[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:04 dist-test-slave-65mx krb5kdc[9410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:04 dist-test-slave-65mx krb5kdc[9410](info): set up 2 sockets
Aug 09 03:25:04 dist-test-slave-65mx krb5kdc[9410](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 03:25:05.237617 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:05 dist-test-slave-65mx krb5kdc[9410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:05.368217 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:38189
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:38189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:05.488642 9425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:05.488919 9425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:05.489005 9425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:05.493273 9425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:05.493331 9425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:05.493352 9425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:05.493369 9425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:05.493549 9425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:05.499326 9425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:38189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:38189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:05.500478 9425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:05.501781 9425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:05.510756 9431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:05.510763 9433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:05.510854 9430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:05.511063 9425 server_base.cc:1034] running on GCE node
I20260809 03:25:05.511672 9425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:05.512980 9425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:05.514176 9425 hybrid_clock.cc:648] HybridClock initialized: now 1786245905514125 us; error 62 us; skew 500 ppm
Aug 09 03:25:05 dist-test-slave-65mx krb5kdc[9410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245905, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:05.517685 9425 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:05.518550 9425 webserver.cc:460] Webserver started at http://127.9.46.254:46237/ using document root <none> and password file <none>
I20260809 03:25:05.519115 9425 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:05.519210 9425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:05.519443 9425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:05.521768 9425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "4a6a5afcdc8645579f9de8fc640b3eb2"
format_stamp: "Formatted at 2026-08-09 03:25:05 on dist-test-slave-65mx"
server_key: "1b9b582a262b78996b05199a9a376aea"
server_key_iv: "4c96ea6e8752d11b363942527fa77aac"
server_key_version: "encryptionkey@0"
I20260809 03:25:05.522347 9425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "4a6a5afcdc8645579f9de8fc640b3eb2"
format_stamp: "Formatted at 2026-08-09 03:25:05 on dist-test-slave-65mx"
server_key: "1b9b582a262b78996b05199a9a376aea"
server_key_iv: "4c96ea6e8752d11b363942527fa77aac"
server_key_version: "encryptionkey@0"
I20260809 03:25:05.526623 9425 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:25:05.529243 9441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:05.530462 9425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:05.530618 9425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "4a6a5afcdc8645579f9de8fc640b3eb2"
format_stamp: "Formatted at 2026-08-09 03:25:05 on dist-test-slave-65mx"
server_key: "1b9b582a262b78996b05199a9a376aea"
server_key_iv: "4c96ea6e8752d11b363942527fa77aac"
server_key_version: "encryptionkey@0"
I20260809 03:25:05.530733 9425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:05.551092 9425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:05.554450 9425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:05.554760 9425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:05.569268 9425 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:38189
I20260809 03:25:05.569281 9498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:38189 every 8 connection(s)
I20260809 03:25:05.570353 9425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:05.573192 9499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:05.575341 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 9425
I20260809 03:25:05.575490 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:05.576959 9403 external_mini_cluster.cc:1428] Setting key 31b172000c0152b3412f33b0b01d40c0
I20260809 03:25:05.579300 9499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2: Bootstrap starting.
I20260809 03:25:05.583256 9499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:05.584961 9499 log.cc:824] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:05.587515 9499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2: No bootstrap required, opened a new log
I20260809 03:25:05.593940 9499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a6a5afcdc8645579f9de8fc640b3eb2" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38189 } }
I20260809 03:25:05.594244 9499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:05.594309 9499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a6a5afcdc8645579f9de8fc640b3eb2, State: Initialized, Role: FOLLOWER
I20260809 03:25:05.594832 9499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a6a5afcdc8645579f9de8fc640b3eb2" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38189 } }
I20260809 03:25:05.594995 9499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:05.595057 9499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:05.595153 9499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:05.596437 9499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a6a5afcdc8645579f9de8fc640b3eb2" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38189 } }
I20260809 03:25:05.596849 9499 leader_election.cc:302] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a6a5afcdc8645579f9de8fc640b3eb2; no voters:
I20260809 03:25:05.597231 9499 leader_election.cc:288] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:05.597386 9504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:05.597771 9504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [term 1 LEADER]: Becoming Leader. State: Replica: 4a6a5afcdc8645579f9de8fc640b3eb2, State: Running, Role: LEADER
I20260809 03:25:05.598232 9504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a6a5afcdc8645579f9de8fc640b3eb2" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38189 } }
I20260809 03:25:05.598846 9499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:05.600281 9505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a6a5afcdc8645579f9de8fc640b3eb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a6a5afcdc8645579f9de8fc640b3eb2" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38189 } } }
I20260809 03:25:05.600418 9505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:05.600612 9506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a6a5afcdc8645579f9de8fc640b3eb2. Latest consensus state: current_term: 1 leader_uuid: "4a6a5afcdc8645579f9de8fc640b3eb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a6a5afcdc8645579f9de8fc640b3eb2" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38189 } } }
I20260809 03:25:05.600735 9506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:05.601336 9509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 03:25:05 dist-test-slave-65mx krb5kdc[9410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:05.605804 9509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:05.610989 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:05.580020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.9.46.254:38189)
0809 03:25:05.580780 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:05.580834 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:05.582046 (+ 1212us) server_negotiation.cc:407] Connection header received
0809 03:25:05.588102 (+ 6056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:05.588119 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:05.588441 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:05.588754 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:05.592683 (+ 3929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:05.593757 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:05.595652 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:05.596161 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:05.596291 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:05.603656 (+ 7365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:05.603663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:05.603678 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:05.603711 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:05.606391 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:05.607267 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:05.607273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:05.607279 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:05.607418 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:05.607929 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:05.607946 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:05.607948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:05.608611 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:05.608825 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:05.609401 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 03:25:05.609679 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":185,"threads_started":1}
I20260809 03:25:05.613720 9509 catalog_manager.cc:1347] Generated new cluster ID: f6fc6298f8684a87939ad552773020f6
I20260809 03:25:05.613785 9509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:05.624130 9509 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:05.625433 9509 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:05.635452 9509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a6a5afcdc8645579f9de8fc640b3eb2: Generated new TSK 0
I20260809 03:25:05.636255 9509 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:25:05.675719 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:38189
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:05.843144 9526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:05.843421 9526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:05.843473 9526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:05.847815 9526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:05.847886 9526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:05.847985 9526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:05.853782 9526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:38189
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:05.854940 9526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:05.856112 9526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:05.868296 9536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:05.868281 9533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:05.868260 9534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:05.868657 9526 server_base.cc:1034] running on GCE node
I20260809 03:25:05.869113 9526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:05.869793 9526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:05.870987 9526 hybrid_clock.cc:648] HybridClock initialized: now 1786245905870951 us; error 50 us; skew 500 ppm
Aug 09 03:25:05 dist-test-slave-65mx krb5kdc[9410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245905, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:05.874688 9526 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:25:05.875622 9526 webserver.cc:460] Webserver started at http://127.9.46.193:44537/ using document root <none> and password file <none>
I20260809 03:25:05.876240 9526 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:05.876338 9526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:05.876621 9526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:05.879014 9526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "b1f5eb1b06094cbb91051f6730d389a2"
format_stamp: "Formatted at 2026-08-09 03:25:05 on dist-test-slave-65mx"
server_key: "af021c2de96af37acd1c29dc0dc42fd0"
server_key_iv: "c4087e2282ab586219dd823c4614bf78"
server_key_version: "encryptionkey@0"
I20260809 03:25:05.879619 9526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "b1f5eb1b06094cbb91051f6730d389a2"
format_stamp: "Formatted at 2026-08-09 03:25:05 on dist-test-slave-65mx"
server_key: "af021c2de96af37acd1c29dc0dc42fd0"
server_key_iv: "c4087e2282ab586219dd823c4614bf78"
server_key_version: "encryptionkey@0"
I20260809 03:25:05.883813 9526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260809 03:25:05.886518 9543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:05.887776 9526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:25:05.887924 9526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "b1f5eb1b06094cbb91051f6730d389a2"
format_stamp: "Formatted at 2026-08-09 03:25:05 on dist-test-slave-65mx"
server_key: "af021c2de96af37acd1c29dc0dc42fd0"
server_key_iv: "c4087e2282ab586219dd823c4614bf78"
server_key_version: "encryptionkey@0"
I20260809 03:25:05.888046 9526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:05.911964 9526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:05.915459 9526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:05.915742 9526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:05.916635 9526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:05.917642 9526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:05.917718 9526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:05.917793 9526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:05.917845 9526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:05.938133 9526 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:43741
I20260809 03:25:05.938236 9665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:43741 every 8 connection(s)
I20260809 03:25:05.939157 9526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:05.944865 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 9526
I20260809 03:25:05.945003 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:05.945322 9403 external_mini_cluster.cc:1428] Setting key 85283607c340d950e73603f627ee05fa
Aug 09 03:25:05 dist-test-slave-65mx krb5kdc[9410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245905, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:05.956717 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:05.941270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:51889 (local address 127.9.46.254:38189)
0809 03:25:05.941411 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:05.941415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:05.942542 (+ 1127us) server_negotiation.cc:407] Connection header received
0809 03:25:05.944180 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:05.944183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:05.944230 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:05.944305 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:05.946337 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:05.946955 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:05.948995 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:05.949257 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:05.949375 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:05.952305 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:05.952312 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:05.952318 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:05.952353 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:05.954139 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:05.954680 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:05.954685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:05.954689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:05.954783 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:05.955193 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:05.955196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:05.955198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:05.955559 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:05.955713 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:05.956258 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 03:25:05.956435 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:25:05.957767 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:05.941664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:38189 (local address 127.9.46.193:51889)
0809 03:25:05.942367 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:05.942414 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:05.943979 (+ 1565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:05.944571 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:05.944580 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:05.945389 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:05.946187 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:05.946207 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:05.947087 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:05.947091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:05.948856 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:05.948871 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:05.949420 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:05.949424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:05.949659 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:05.950382 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:05.950398 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:05.952129 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:05.954271 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:05.954275 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:05.954287 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:05.954568 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:05.954926 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:05.954929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:05.954930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:05.955075 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:05.955723 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:05.955728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:05.955964 (+ 236us) client_negotiation.cc:769] Sending connection context
0809 03:25:05.956148 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 03:25:05.956529 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":185,"threads_started":1}
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:25:05.959420 9666 heartbeater.cc:347] Connected to a master server at 127.9.46.254:38189
I20260809 03:25:05.959681 9666 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:05.960189 9666 heartbeater.cc:511] Master 127.9.46.254:38189 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
I20260809 03:25:05.961889 9459 ts_manager.cc:194] Registered new tserver with Master: b1f5eb1b06094cbb91051f6730d389a2 (127.9.46.193:43741)
I20260809 03:25:05.963061 9459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:51889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:25:05.979902 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:38189
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:06.143301 9671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:06.143561 9671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:06.143642 9671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:06.147910 9671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:06.147971 9671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:06.148069 9671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:06.153648 9671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:38189
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:06.154685 9671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:06.155726 9671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:06.167081 9679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:06.167322 9677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:06.167322 9676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:06.167665 9671 server_base.cc:1034] running on GCE node
I20260809 03:25:06.168041 9671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:06.168776 9671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:06.169940 9671 hybrid_clock.cc:648] HybridClock initialized: now 1786245906169926 us; error 27 us; skew 500 ppm
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245906, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:06.173365 9671 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:25:06.174269 9671 webserver.cc:460] Webserver started at http://127.9.46.194:42387/ using document root <none> and password file <none>
I20260809 03:25:06.174814 9671 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:06.174907 9671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:06.175151 9671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:06.177398 9671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "a02fe50a6b754906a20c5946ced1bdc5"
format_stamp: "Formatted at 2026-08-09 03:25:06 on dist-test-slave-65mx"
server_key: "a5e53a192891ad5aece7f8454347862c"
server_key_iv: "8adea8f310df81676ec0c0d4613097f9"
server_key_version: "encryptionkey@0"
I20260809 03:25:06.178062 9671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "a02fe50a6b754906a20c5946ced1bdc5"
format_stamp: "Formatted at 2026-08-09 03:25:06 on dist-test-slave-65mx"
server_key: "a5e53a192891ad5aece7f8454347862c"
server_key_iv: "8adea8f310df81676ec0c0d4613097f9"
server_key_version: "encryptionkey@0"
I20260809 03:25:06.181993 9671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:25:06.184703 9687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:06.185853 9671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:25:06.185986 9671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "a02fe50a6b754906a20c5946ced1bdc5"
format_stamp: "Formatted at 2026-08-09 03:25:06 on dist-test-slave-65mx"
server_key: "a5e53a192891ad5aece7f8454347862c"
server_key_iv: "8adea8f310df81676ec0c0d4613097f9"
server_key_version: "encryptionkey@0"
I20260809 03:25:06.186096 9671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:06.216475 9671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:06.220111 9671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:06.220402 9671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:06.221253 9671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:06.222363 9671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:06.222442 9671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:06.222509 9671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:06.222558 9671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:06.243238 9671 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:40571
I20260809 03:25:06.243381 9809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:40571 every 8 connection(s)
I20260809 03:25:06.244200 9671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:06.249164 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 9671
I20260809 03:25:06.249279 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:06.249552 9403 external_mini_cluster.cc:1428] Setting key 8fcf103302bb8770c6cdd26f696dac06
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245906, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:06.260497 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:06.246343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:37949 (local address 127.9.46.254:38189)
0809 03:25:06.246529 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:06.246534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:06.247518 (+ 984us) server_negotiation.cc:407] Connection header received
0809 03:25:06.249183 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:06.249190 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:06.249255 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:06.249349 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:06.250910 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.251460 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.252851 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.253073 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.253183 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:06.256235 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:06.256242 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:06.256249 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:06.256286 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:06.258149 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:06.258650 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:06.258656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:06.258661 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:06.258763 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:06.259166 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:06.259169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:06.259171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:06.259463 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:06.259649 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:06.260113 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 03:25:06.260287 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 03:25:06.261243 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:06.246715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:38189 (local address 127.9.46.194:37949)
0809 03:25:06.247360 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:06.247404 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:06.248920 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:06.249553 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:06.249560 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:06.250042 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:06.250759 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:06.250784 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.251596 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.251599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:06.252718 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:06.252727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.253224 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.253228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:06.253456 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:06.254228 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:06.254245 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:06.256052 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:06.258272 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:06.258277 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:06.258288 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:06.258544 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:06.258873 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:06.258875 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:06.258876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:06.259008 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:06.259627 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:06.259632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:06.259858 (+ 226us) client_negotiation.cc:769] Sending connection context
0809 03:25:06.260012 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 03:25:06.260275 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":207,"threads_started":1}
I20260809 03:25:06.262575 9810 heartbeater.cc:347] Connected to a master server at 127.9.46.254:38189
I20260809 03:25:06.262888 9810 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:25:06.263582 9810 heartbeater.cc:511] Master 127.9.46.254:38189 requested a full tablet report, sending...
I20260809 03:25:06.264684 9459 ts_manager.cc:194] Registered new tserver with Master: a02fe50a6b754906a20c5946ced1bdc5 (127.9.46.194:40571)
I20260809 03:25:06.265249 9459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:37949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:25:06.284147 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:38189
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:06.413168 9816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:06.413425 9816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:06.413480 9816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:06.417484 9816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:06.417533 9816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:06.417593 9816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:06.422806 9816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:38189
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:06.423812 9816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:06.424885 9816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:06.435942 9824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:06.436080 9822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:06.435978 9816 server_base.cc:1034] running on GCE node
W20260809 03:25:06.436126 9821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:06.436481 9816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:06.437196 9816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:06.438359 9816 hybrid_clock.cc:648] HybridClock initialized: now 1786245906438344 us; error 24 us; skew 500 ppm
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245906, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:06.441635 9816 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:25:06.442446 9816 webserver.cc:460] Webserver started at http://127.9.46.195:36275/ using document root <none> and password file <none>
I20260809 03:25:06.442998 9816 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:06.443090 9816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:06.443317 9816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:06.445592 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "fb6c3dc9fb3145f7811f36a7bb4a4608"
format_stamp: "Formatted at 2026-08-09 03:25:06 on dist-test-slave-65mx"
server_key: "4346b7815356bed50b05d3b93ff95560"
server_key_iv: "4e593675ae324a35d4085c4566f17e13"
server_key_version: "encryptionkey@0"
I20260809 03:25:06.446169 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "fb6c3dc9fb3145f7811f36a7bb4a4608"
format_stamp: "Formatted at 2026-08-09 03:25:06 on dist-test-slave-65mx"
server_key: "4346b7815356bed50b05d3b93ff95560"
server_key_iv: "4e593675ae324a35d4085c4566f17e13"
server_key_version: "encryptionkey@0"
I20260809 03:25:06.450513 9816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:25:06.453156 9832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:06.454331 9816 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:06.454465 9816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "fb6c3dc9fb3145f7811f36a7bb4a4608"
format_stamp: "Formatted at 2026-08-09 03:25:06 on dist-test-slave-65mx"
server_key: "4346b7815356bed50b05d3b93ff95560"
server_key_iv: "4e593675ae324a35d4085c4566f17e13"
server_key_version: "encryptionkey@0"
I20260809 03:25:06.454578 9816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:06.470711 9816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:06.474387 9816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:06.474648 9816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:06.475370 9816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:06.476289 9816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:06.476364 9816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:06.476433 9816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:06.476483 9816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:06.495827 9816 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:44119
I20260809 03:25:06.495853 9955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:44119 every 8 connection(s)
I20260809 03:25:06.496810 9816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:06.501355 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 9816
I20260809 03:25:06.501497 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:06.501807 9403 external_mini_cluster.cc:1428] Setting key 696c9dab797c94ff212ff99315d37f4a
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245906, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:06.512367 9501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:06.498865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:44125 (local address 127.9.46.254:38189)
0809 03:25:06.499029 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:06.499033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:06.500254 (+ 1221us) server_negotiation.cc:407] Connection header received
0809 03:25:06.502170 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:06.502173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:06.502235 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:06.502337 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:06.503772 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.504599 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.505906 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.506149 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.506249 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:06.509089 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:06.509092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:06.509094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:06.509118 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:06.510552 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:06.511043 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:06.511046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:06.511047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:06.511123 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:06.511431 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:06.511434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:06.511435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:06.511608 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:06.511685 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:06.512149 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 03:25:06.512244 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:25:06.513267 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:06.499229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:38189 (local address 127.9.46.195:44125)
0809 03:25:06.500014 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:06.500065 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:06.501932 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:06.502552 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:06.502561 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:06.503050 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:06.503623 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:06.503639 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.504736 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.504740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:06.505741 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:06.505751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.506296 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.506299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:06.506447 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:06.507260 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:06.507276 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:06.508925 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:06.510667 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:06.510672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:06.510683 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:06.510927 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:06.511210 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:06.511212 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:06.511214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:06.511341 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:06.511700 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:06.511705 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:06.511920 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 03:25:06.512069 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 03:25:06.512307 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":203,"threads_started":1}
I20260809 03:25:06.514644 9956 heartbeater.cc:347] Connected to a master server at 127.9.46.254:38189
I20260809 03:25:06.514904 9956 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:06.515456 9956 heartbeater.cc:511] Master 127.9.46.254:38189 requested a full tablet report, sending...
I20260809 03:25:06.516574 9458 ts_manager.cc:194] Registered new tserver with Master: fb6c3dc9fb3145f7811f36a7bb4a4608 (127.9.46.195:44119)
I20260809 03:25:06.517249 9458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:44125
I20260809 03:25:06.528304 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245906, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245906, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:25:06.558058 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:06.548403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48896 (local address 127.9.46.193:43741)
0809 03:25:06.548797 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:06.548804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:06.548829 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:25:06.548897 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:06.548902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:06.549086 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:06.549209 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:06.549944 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.550734 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.551588 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.551984 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.552066 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:06.554765 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:06.554775 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:06.554788 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:06.554814 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:06.556329 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:06.556811 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:06.556817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:06.556818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:06.556904 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:06.557146 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:06.557148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:06.557149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:06.557401 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:06.557559 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:06.557726 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 03:25:06.557856 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":160,"threads_started":1}
W20260809 03:25:06.559089 9574 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48896
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:25:06.587695 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:06.579063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.9.46.193:43741)
0809 03:25:06.579226 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:06.579231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:06.579391 (+ 160us) server_negotiation.cc:407] Connection header received
0809 03:25:06.579597 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:06.579600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:06.579647 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:06.579732 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:06.580443 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.581057 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.581943 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:06.582183 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:06.582296 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:06.584701 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:06.584705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:06.584707 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:06.584732 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:06.586018 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:06.586447 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:06.586458 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:06.586459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:06.586543 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:06.586830 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:06.586835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:06.586837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:06.587010 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:06.587173 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:06.587397 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 03:25:06.587533 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 03:25:06.588805 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 9526
I20260809 03:25:06.596655 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 9671
I20260809 03:25:06.604444 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 9816
I20260809 03:25:06.613269 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 9425
2026-08-09T03:25:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2478 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9971](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9971](info): set up 2 sockets
Aug 09 03:25:06 dist-test-slave-65mx krb5kdc[9971](info): commencing operation
krb5kdc: starting...
W20260809 03:25:07.701963 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:07 dist-test-slave-65mx krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:07.831686 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:44289
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:44289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:07.943552 9986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:07.943830 9986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:07.943871 9986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:07.948014 9986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:07.948086 9986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:07.948107 9986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:07.948123 9986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:07.948138 9986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:07.953611 9986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:44289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:44289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:07.954695 9986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:07.955749 9986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:07.964574 9991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:07.964581 9992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:07.964797 9986 server_base.cc:1034] running on GCE node
W20260809 03:25:07.964632 9994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:07.965433 9986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:07.966499 9986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:07.967707 9986 hybrid_clock.cc:648] HybridClock initialized: now 1786245907967687 us; error 31 us; skew 500 ppm
Aug 09 03:25:07 dist-test-slave-65mx krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245907, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:07.975383 9986 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:07.976259 9986 webserver.cc:460] Webserver started at http://127.9.46.254:34417/ using document root <none> and password file <none>
I20260809 03:25:07.976876 9986 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:07.976939 9986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:07.977123 9986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:07.979311 9986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "b081f0e493cc4dd8a4d724bad745fdcd"
format_stamp: "Formatted at 2026-08-09 03:25:07 on dist-test-slave-65mx"
server_key: "95cea29fedcb1d6cda5f7fe80903b3fd"
server_key_iv: "3a648088b260c5be6e9deda9f1e0a702"
server_key_version: "encryptionkey@0"
I20260809 03:25:07.979823 9986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "b081f0e493cc4dd8a4d724bad745fdcd"
format_stamp: "Formatted at 2026-08-09 03:25:07 on dist-test-slave-65mx"
server_key: "95cea29fedcb1d6cda5f7fe80903b3fd"
server_key_iv: "3a648088b260c5be6e9deda9f1e0a702"
server_key_version: "encryptionkey@0"
I20260809 03:25:07.983619 9986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 03:25:07.986027 10001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:07.987283 9986 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:25:07.987389 9986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "b081f0e493cc4dd8a4d724bad745fdcd"
format_stamp: "Formatted at 2026-08-09 03:25:07 on dist-test-slave-65mx"
server_key: "95cea29fedcb1d6cda5f7fe80903b3fd"
server_key_iv: "3a648088b260c5be6e9deda9f1e0a702"
server_key_version: "encryptionkey@0"
I20260809 03:25:07.987476 9986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:08.015151 9986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:08.018433 9986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:08.018589 9986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:08.025722 9986 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:44289
I20260809 03:25:08.025751 10059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:44289 every 8 connection(s)
I20260809 03:25:08.026652 9986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:08.029496 10061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:08.029700 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 9986
I20260809 03:25:08.029825 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:08.030085 9403 external_mini_cluster.cc:1428] Setting key bfe488b5c7e13746f07555c2232999d7
I20260809 03:25:08.036818 10061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd: Bootstrap starting.
I20260809 03:25:08.040081 10061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd: Neither blocks nor log segments found. Creating new log.
Aug 09 03:25:08 dist-test-slave-65mx krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:08.041075 10061 log.cc:824] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:08.043692 10061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd: No bootstrap required, opened a new log
I20260809 03:25:08.047209 10064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.031440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.9.46.254:44289)
0809 03:25:08.031876 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:08.031890 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:08.031924 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:25:08.032778 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:08.032797 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:08.033134 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:08.033458 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:08.034317 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.035873 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.037233 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.038041 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.041591 (+ 3550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.041831 (+ 240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:08.041839 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:08.041858 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:08.041890 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:08.044192 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:08.044797 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:08.044806 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:08.044811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:08.044934 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:08.045305 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:08.045308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:08.045310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:08.045693 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:08.045909 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:08.046198 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 03:25:08.046479 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
I20260809 03:25:08.047472 10061 raft_consensus.cc:348] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b081f0e493cc4dd8a4d724bad745fdcd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44289 } }
I20260809 03:25:08.047731 10061 raft_consensus.cc:374] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:08.047812 10061 raft_consensus.cc:729] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b081f0e493cc4dd8a4d724bad745fdcd, State: Initialized, Role: FOLLOWER
I20260809 03:25:08.048336 10061 consensus_queue.cc:260] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b081f0e493cc4dd8a4d724bad745fdcd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44289 } }
I20260809 03:25:08.048501 10061 raft_consensus.cc:388] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:08.048583 10061 raft_consensus.cc:482] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:08.048682 10061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:08.049887 10061 raft_consensus.cc:504] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b081f0e493cc4dd8a4d724bad745fdcd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44289 } }
I20260809 03:25:08.050179 10061 leader_election.cc:302] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b081f0e493cc4dd8a4d724bad745fdcd; no voters:
I20260809 03:25:08.050413 10061 leader_election.cc:288] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:08.050558 10066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:08.050781 10066 raft_consensus.cc:686] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [term 1 LEADER]: Becoming Leader. State: Replica: b081f0e493cc4dd8a4d724bad745fdcd, State: Running, Role: LEADER
I20260809 03:25:08.051087 10066 consensus_queue.cc:237] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b081f0e493cc4dd8a4d724bad745fdcd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44289 } }
I20260809 03:25:08.051712 10061 sys_catalog.cc:564] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:08.052268 10068 sys_catalog.cc:455] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b081f0e493cc4dd8a4d724bad745fdcd. Latest consensus state: current_term: 1 leader_uuid: "b081f0e493cc4dd8a4d724bad745fdcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b081f0e493cc4dd8a4d724bad745fdcd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44289 } } }
I20260809 03:25:08.052484 10068 sys_catalog.cc:458] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:08.052745 10067 sys_catalog.cc:455] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b081f0e493cc4dd8a4d724bad745fdcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b081f0e493cc4dd8a4d724bad745fdcd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44289 } } }
I20260809 03:25:08.052865 10067 sys_catalog.cc:458] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:08.053107 10076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:08.056196 10076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:08.061633 10076 catalog_manager.cc:1347] Generated new cluster ID: fb7a2c2e01854bc3b7bfbdefacdb7114
I20260809 03:25:08.061703 10076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:08.075493 10076 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:08.076464 10076 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:08.082908 10076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b081f0e493cc4dd8a4d724bad745fdcd: Generated new TSK 0
I20260809 03:25:08.083570 10076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:25:08.120337 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44289
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:08.232388 10091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:08.232707 10091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:08.232755 10091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:08.236716 10091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:08.236765 10091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:08.236820 10091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:08.241916 10091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44289
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:08.242864 10091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:08.243789 10091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:08.253125 10100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:08.253208 10096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:08.253217 10098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:08.253297 10091 server_base.cc:1034] running on GCE node
I20260809 03:25:08.253839 10091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:08.254488 10091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:08.255849 10091 hybrid_clock.cc:648] HybridClock initialized: now 1786245908255823 us; error 46 us; skew 500 ppm
Aug 09 03:25:08 dist-test-slave-65mx krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:08.258831 10091 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:25:08.259748 10091 webserver.cc:460] Webserver started at http://127.9.46.193:43211/ using document root <none> and password file <none>
I20260809 03:25:08.260388 10091 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:08.260511 10091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:08.260790 10091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:08.262981 10091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd"
format_stamp: "Formatted at 2026-08-09 03:25:08 on dist-test-slave-65mx"
server_key: "d33db36cc860c2f80487371cecb183da"
server_key_iv: "5e318eb567b263ce8373c5400e0f6611"
server_key_version: "encryptionkey@0"
I20260809 03:25:08.263598 10091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd"
format_stamp: "Formatted at 2026-08-09 03:25:08 on dist-test-slave-65mx"
server_key: "d33db36cc860c2f80487371cecb183da"
server_key_iv: "5e318eb567b263ce8373c5400e0f6611"
server_key_version: "encryptionkey@0"
I20260809 03:25:08.267611 10091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 03:25:08.270112 10108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:08.271361 10091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:25:08.271482 10091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd"
format_stamp: "Formatted at 2026-08-09 03:25:08 on dist-test-slave-65mx"
server_key: "d33db36cc860c2f80487371cecb183da"
server_key_iv: "5e318eb567b263ce8373c5400e0f6611"
server_key_version: "encryptionkey@0"
I20260809 03:25:08.271592 10091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:08.293347 10091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:08.296250 10091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:08.296453 10091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:08.297101 10091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:08.297931 10091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:08.297992 10091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:08.298082 10091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:08.298117 10091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:08.307479 10091 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:46619
I20260809 03:25:08.307510 10232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:46619 every 8 connection(s)
I20260809 03:25:08.308378 10091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:08.317425 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 10091
I20260809 03:25:08.317551 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
Aug 09 03:25:08 dist-test-slave-65mx krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:08.317806 9403 external_mini_cluster.cc:1428] Setting key f9179946e24ae8d22ead1d36c69ba9f0
I20260809 03:25:08.322240 10064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.310174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:60989 (local address 127.9.46.254:44289)
0809 03:25:08.310306 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:08.310310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:08.311139 (+ 829us) server_negotiation.cc:407] Connection header received
0809 03:25:08.312461 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:08.312464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:08.312522 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:08.312632 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:08.313770 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.314315 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.315436 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.315682 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.315787 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.318515 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:08.318519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:08.318522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:08.318546 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:08.320093 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:08.320673 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:08.320676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:08.320678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:08.320765 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:08.321118 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:08.321121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:08.321123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:08.321313 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:08.321428 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:08.321924 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 03:25:08.322067 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:25:08.323088 10235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.310570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44289 (local address 127.9.46.193:60989)
0809 03:25:08.310990 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:08.311033 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:08.312254 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:08.312808 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:08.312815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:08.313171 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:08.313625 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.313638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.314447 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.314456 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.315303 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.315312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.315839 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.315844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.316004 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.316647 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:08.316663 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:08.318319 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:08.320254 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:08.320259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:08.320273 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:08.320577 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:08.320859 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:08.320862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:08.320865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:08.321010 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:08.321428 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:08.321436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:08.321682 (+ 246us) client_negotiation.cc:769] Sending connection context
0809 03:25:08.321880 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 03:25:08.322196 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":108,"threads_started":1}
I20260809 03:25:08.324460 10233 heartbeater.cc:347] Connected to a master server at 127.9.46.254:44289
I20260809 03:25:08.324723 10233 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:08.325270 10233 heartbeater.cc:511] Master 127.9.46.254:44289 requested a full tablet report, sending...
I20260809 03:25:08.326860 10020 ts_manager.cc:194] Registered new tserver with Master: 8c9d8c6e4c2248ecac9f6eb8156770dd (127.9.46.193:46619)
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:25:08.327996 10020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:60989
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:25:08.348013 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44289
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:08.466338 10239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:08.466591 10239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:08.466636 10239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:08.470553 10239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:08.470608 10239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:08.470666 10239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:08.475972 10239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44289
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:08.476981 10239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:08.477923 10239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:08.487824 10247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:08.487830 10245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:08.487854 10244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:08.488087 10239 server_base.cc:1034] running on GCE node
I20260809 03:25:08.488674 10239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:08.489300 10239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:08.490489 10239 hybrid_clock.cc:648] HybridClock initialized: now 1786245908490482 us; error 34 us; skew 500 ppm
Aug 09 03:25:08 dist-test-slave-65mx krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:08.493362 10239 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:25:08.494251 10239 webserver.cc:460] Webserver started at http://127.9.46.194:36719/ using document root <none> and password file <none>
I20260809 03:25:08.494869 10239 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:08.494959 10239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:08.495191 10239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:08.497463 10239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "4b36a411f680487cb32d0dabe5b8c1ee"
format_stamp: "Formatted at 2026-08-09 03:25:08 on dist-test-slave-65mx"
server_key: "ed79ed39506b1cb45606060cd6378873"
server_key_iv: "436e318bf3b00ccb3d523aaa2b756c7e"
server_key_version: "encryptionkey@0"
I20260809 03:25:08.498030 10239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "4b36a411f680487cb32d0dabe5b8c1ee"
format_stamp: "Formatted at 2026-08-09 03:25:08 on dist-test-slave-65mx"
server_key: "ed79ed39506b1cb45606060cd6378873"
server_key_iv: "436e318bf3b00ccb3d523aaa2b756c7e"
server_key_version: "encryptionkey@0"
I20260809 03:25:08.501709 10239 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 03:25:08.504172 10255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:08.505498 10239 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:25:08.505645 10239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "4b36a411f680487cb32d0dabe5b8c1ee"
format_stamp: "Formatted at 2026-08-09 03:25:08 on dist-test-slave-65mx"
server_key: "ed79ed39506b1cb45606060cd6378873"
server_key_iv: "436e318bf3b00ccb3d523aaa2b756c7e"
server_key_version: "encryptionkey@0"
I20260809 03:25:08.505759 10239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:08.522382 10239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:08.525700 10239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:08.525899 10239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:08.526486 10239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:08.527308 10239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:08.527380 10239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:08.527449 10239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:08.527501 10239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:08.537065 10239 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:35349
I20260809 03:25:08.537119 10377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:35349 every 8 connection(s)
I20260809 03:25:08.538010 10239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:08.545693 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 10239
I20260809 03:25:08.545826 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:08.546164 9403 external_mini_cluster.cc:1428] Setting key c753c7137a41369e7c2c2c26fc1da259
Aug 09 03:25:08 dist-test-slave-65mx krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:08.552264 10064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.539791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:35435 (local address 127.9.46.254:44289)
0809 03:25:08.539943 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:08.539946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:08.540767 (+ 821us) server_negotiation.cc:407] Connection header received
0809 03:25:08.542062 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:08.542065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:08.542124 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:08.542252 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:08.543369 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.543896 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.545107 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.545331 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.545438 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.548406 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:08.548410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:08.548425 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:08.548453 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:08.550096 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:08.550646 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:08.550651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:08.550653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:08.550740 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:08.551088 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:08.551094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:08.551098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:08.551329 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:08.551439 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:08.551929 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 03:25:08.552066 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:25:08.553213 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.540133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44289 (local address 127.9.46.194:35435)
0809 03:25:08.540623 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:08.540667 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:08.541852 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:08.542418 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:08.542424 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:08.542771 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:08.543229 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.543241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.544027 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.544030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.544979 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.544989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.545488 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.545492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.545666 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.546366 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:08.546384 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:08.548220 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:08.550231 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:08.550237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:08.550252 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:08.550516 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:08.550836 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:08.550840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:08.550842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:08.550986 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:08.551436 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:08.551442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:08.551691 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 03:25:08.551886 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 03:25:08.552213 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":117,"threads_started":1}
I20260809 03:25:08.554423 10378 heartbeater.cc:347] Connected to a master server at 127.9.46.254:44289
I20260809 03:25:08.554699 10378 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:08.555285 10378 heartbeater.cc:511] Master 127.9.46.254:44289 requested a full tablet report, sending...
I20260809 03:25:08.556453 10020 ts_manager.cc:194] Registered new tserver with Master: 4b36a411f680487cb32d0dabe5b8c1ee (127.9.46.194:35349)
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:25:08.557340 10020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:35435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:25:08.576714 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44289
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:08.688925 10383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:08.689174 10383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:08.689218 10383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:08.693331 10383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:08.693380 10383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:08.693439 10383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:08.698639 10383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44289
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:08.699577 10383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:08.700501 10383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:08.710094 10388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:08.710207 10391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:08.710194 10383 server_base.cc:1034] running on GCE node
W20260809 03:25:08.710193 10389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:08.710734 10383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:08.711398 10383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:08.712634 10383 hybrid_clock.cc:648] HybridClock initialized: now 1786245908712613 us; error 25 us; skew 500 ppm
Aug 09 03:25:08 dist-test-slave-65mx krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:08.715350 10383 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:25:08.716183 10383 webserver.cc:460] Webserver started at http://127.9.46.195:38227/ using document root <none> and password file <none>
I20260809 03:25:08.716777 10383 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:08.716841 10383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:08.717022 10383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:08.718961 10383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "9a0c618dedc44df897c90bd71f766472"
format_stamp: "Formatted at 2026-08-09 03:25:08 on dist-test-slave-65mx"
server_key: "4273c4de8506c833f4a14efab6869917"
server_key_iv: "c47baf22e85dce74b9cf47dc27045853"
server_key_version: "encryptionkey@0"
I20260809 03:25:08.719461 10383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "9a0c618dedc44df897c90bd71f766472"
format_stamp: "Formatted at 2026-08-09 03:25:08 on dist-test-slave-65mx"
server_key: "4273c4de8506c833f4a14efab6869917"
server_key_iv: "c47baf22e85dce74b9cf47dc27045853"
server_key_version: "encryptionkey@0"
I20260809 03:25:08.723079 10383 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 03:25:08.725536 10398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:08.726835 10383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:08.727049 10383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "9a0c618dedc44df897c90bd71f766472"
format_stamp: "Formatted at 2026-08-09 03:25:08 on dist-test-slave-65mx"
server_key: "4273c4de8506c833f4a14efab6869917"
server_key_iv: "c47baf22e85dce74b9cf47dc27045853"
server_key_version: "encryptionkey@0"
I20260809 03:25:08.727165 10383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:08.743418 10383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:08.746753 10383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:08.746965 10383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:08.747601 10383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:08.748433 10383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:08.748481 10383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:08.748584 10383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:08.748624 10383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:08.758204 10383 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:46689
I20260809 03:25:08.758229 10521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:46689 every 8 connection(s)
I20260809 03:25:08.759105 10383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:08.764096 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 10383
I20260809 03:25:08.764235 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:08.764516 9403 external_mini_cluster.cc:1428] Setting key 6859eef4af2ce219de8b64d09cacb33d
Aug 09 03:25:08 dist-test-slave-65mx krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:08.772903 10064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.760985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:34631 (local address 127.9.46.254:44289)
0809 03:25:08.761197 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:08.761202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:08.761933 (+ 731us) server_negotiation.cc:407] Connection header received
0809 03:25:08.763196 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:08.763199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:08.763244 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:08.763318 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:08.764579 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.765193 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.766744 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.766962 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.767048 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.769762 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:08.769765 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:08.769767 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:08.769790 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:08.770998 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:08.771464 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:08.771467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:08.771468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:08.771562 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:08.771895 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:08.771898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:08.771899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:08.772084 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:08.772174 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:08.772679 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 03:25:08.772783 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260809 03:25:08.773801 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.761357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44289 (local address 127.9.46.195:34631)
0809 03:25:08.761792 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:08.761833 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:08.763005 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:08.763484 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:08.763492 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:08.763825 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:08.764389 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.764405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.765301 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.765304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.766617 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.766629 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.767103 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.767106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.767243 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.767918 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:08.767935 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:08.769597 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:08.771117 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:08.771121 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:08.771131 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:08.771375 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:08.771645 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:08.771648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:08.771649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:08.771783 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:08.772187 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:08.772192 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:08.772405 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 03:25:08.772628 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 03:25:08.772873 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":147,"threads_started":1}
I20260809 03:25:08.775070 10522 heartbeater.cc:347] Connected to a master server at 127.9.46.254:44289
I20260809 03:25:08.775343 10522 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:08.775825 10522 heartbeater.cc:511] Master 127.9.46.254:44289 requested a full tablet report, sending...
I20260809 03:25:08.776906 10020 ts_manager.cc:194] Registered new tserver with Master: 9a0c618dedc44df897c90bd71f766472 (127.9.46.195:46689)
I20260809 03:25:08.777536 10020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:34631
I20260809 03:25:08.778730 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:25:08.789381 10064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.782236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.9.46.254:44289)
0809 03:25:08.782346 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:08.782350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:08.782548 (+ 198us) server_negotiation.cc:407] Connection header received
0809 03:25:08.782645 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:08.782647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:08.782684 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:08.782750 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:08.783396 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.783871 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.784695 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.784991 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.785105 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.786397 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:08.786400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:08.786402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:08.786422 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:08.787633 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:08.788005 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:08.788007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:08.788038 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:08.788111 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:08.788392 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:08.788395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:08.788396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:08.788735 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:08.788868 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:08.789131 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 03:25:08.789241 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:25:08.794041 10020 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:08.797278 10020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:08.812424 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.806585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:46619 (local address 127.0.0.1:35604)
0809 03:25:08.808078 (+ 1493us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:08.808092 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:08.808188 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:08.808790 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:08.808793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:08.808813 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:08.808902 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.808909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.810380 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.810383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.811199 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.811208 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.812017 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.812019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.812094 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.812124 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 03:25:08.812208 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:25:08.812297 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1433,"mutex_wait_us":6,"thread_start_us":105,"threads_started":1}
I20260809 03:25:08.812649 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.806730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.9.46.193:46619)
0809 03:25:08.807140 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:08.807157 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:08.808286 (+ 1129us) server_negotiation.cc:407] Connection header received
0809 03:25:08.808352 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:08.808355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:08.808481 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:08.808758 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:08.809031 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.810223 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.811326 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.811916 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.812039 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.812193 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:08.812243 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 03:25:08.812333 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":95,"threads_started":1}
I20260809 03:25:08.813575 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.805734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:46689 (local address 127.0.0.1:56794)
0809 03:25:08.807039 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:08.807094 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:08.807241 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:08.807814 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:08.807819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:08.807841 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:08.808110 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.808124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.810099 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.810106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.811804 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.811810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.813125 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.813131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.813220 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.813238 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 03:25:08.813324 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 03:25:08.813401 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"mutex_wait_us":118,"spinlock_wait_cycles":3968,"thread_start_us":130,"threads_started":1}
I20260809 03:25:08.813954 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.805862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56794 (local address 127.9.46.195:46689)
0809 03:25:08.807410 (+ 1548us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:08.807421 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:08.807439 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:25:08.807497 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:08.807501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:08.807664 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:08.807826 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:08.808272 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.809771 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.811938 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.812805 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.812916 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.813054 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:08.813752 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 03:25:08.813842 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1431,"thread_start_us":256,"threads_started":1}
I20260809 03:25:08.815478 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.805757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:35349 (local address 127.0.0.1:40658)
0809 03:25:08.807080 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:08.807095 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:08.807241 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:08.807839 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:08.807842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:08.807869 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:08.808048 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.808056 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.810166 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.810169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.811796 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:08.811808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.815072 (+ 3264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.815076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:08.815170 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.815191 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:25:08.815278 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 03:25:08.815353 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"mutex_wait_us":218,"spinlock_wait_cycles":6144,"thread_start_us":86,"threads_started":1}
I20260809 03:25:08.815621 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:08.806165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.9.46.194:35349)
0809 03:25:08.806904 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:08.806949 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:08.807417 (+ 468us) server_negotiation.cc:407] Connection header received
0809 03:25:08.807490 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:08.807494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:08.807661 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:08.807860 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:08.808160 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.809926 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.814153 (+ 4227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:08.814940 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:08.815069 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:08.815205 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:08.815347 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 03:25:08.815439 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":319,"threads_started":1}
I20260809 03:25:08.816015 10161 tablet_service.cc:1467] Processing CreateTablet for tablet 2c513ccad67147efabc42b2490fab7df (DEFAULT_TABLE table=test-table [id=b508ecf6dd0f4efda70d379796aaf3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:08.816851 10451 tablet_service.cc:1467] Processing CreateTablet for tablet 2c513ccad67147efabc42b2490fab7df (DEFAULT_TABLE table=test-table [id=b508ecf6dd0f4efda70d379796aaf3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:08.817514 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c513ccad67147efabc42b2490fab7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:08.817956 10451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c513ccad67147efabc42b2490fab7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:08.818441 10306 tablet_service.cc:1467] Processing CreateTablet for tablet 2c513ccad67147efabc42b2490fab7df (DEFAULT_TABLE table=test-table [id=b508ecf6dd0f4efda70d379796aaf3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:08.819362 10306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c513ccad67147efabc42b2490fab7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:08.826545 10541 tablet_bootstrap.cc:492] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee: Bootstrap starting.
I20260809 03:25:08.827306 10544 tablet_bootstrap.cc:492] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472: Bootstrap starting.
I20260809 03:25:08.828859 10543 tablet_bootstrap.cc:492] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd: Bootstrap starting.
I20260809 03:25:08.830564 10544 tablet_bootstrap.cc:654] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:08.831065 10541 tablet_bootstrap.cc:654] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:08.831614 10543 tablet_bootstrap.cc:654] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:08.832808 10541 log.cc:824] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:08.832808 10544 log.cc:824] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:08.833204 10543 log.cc:824] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:08.835184 10541 tablet_bootstrap.cc:492] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee: No bootstrap required, opened a new log
I20260809 03:25:08.835333 10543 tablet_bootstrap.cc:492] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd: No bootstrap required, opened a new log
I20260809 03:25:08.835382 10544 tablet_bootstrap.cc:492] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472: No bootstrap required, opened a new log
I20260809 03:25:08.835407 10541 ts_tablet_manager.cc:1397] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 03:25:08.835589 10543 ts_tablet_manager.cc:1397] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 03:25:08.835589 10544 ts_tablet_manager.cc:1397] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.008s
I20260809 03:25:08.841409 10541 raft_consensus.cc:348] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0c618dedc44df897c90bd71f766472" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 46689 } } peers { permanent_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 35349 } } peers { permanent_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 46619 } }
I20260809 03:25:08.841835 10541 raft_consensus.cc:374] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:08.841920 10541 raft_consensus.cc:729] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b36a411f680487cb32d0dabe5b8c1ee, State: Initialized, Role: FOLLOWER
I20260809 03:25:08.842549 10541 consensus_queue.cc:260] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0c618dedc44df897c90bd71f766472" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 46689 } } peers { permanent_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 35349 } } peers { permanent_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 46619 } }
I20260809 03:25:08.842759 10544 raft_consensus.cc:348] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0c618dedc44df897c90bd71f766472" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 46689 } } peers { permanent_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 35349 } } peers { permanent_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 46619 } }
I20260809 03:25:08.843119 10544 raft_consensus.cc:374] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:08.843235 10544 raft_consensus.cc:729] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0c618dedc44df897c90bd71f766472, State: Initialized, Role: FOLLOWER
I20260809 03:25:08.843648 10378 heartbeater.cc:503] Master 127.9.46.254:44289 was elected leader, sending a full tablet report...
I20260809 03:25:08.843806 10544 consensus_queue.cc:260] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0c618dedc44df897c90bd71f766472" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 46689 } } peers { permanent_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 35349 } } peers { permanent_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 46619 } }
I20260809 03:25:08.844213 10543 raft_consensus.cc:348] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0c618dedc44df897c90bd71f766472" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 46689 } } peers { permanent_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 35349 } } peers { permanent_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 46619 } }
I20260809 03:25:08.844444 10543 raft_consensus.cc:374] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:08.844488 10543 raft_consensus.cc:729] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9d8c6e4c2248ecac9f6eb8156770dd, State: Initialized, Role: FOLLOWER
I20260809 03:25:08.844785 10544 ts_tablet_manager.cc:1428] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472: Time spent starting tablet: real 0.009s user 0.009s sys 0.001s
I20260809 03:25:08.843880 10541 ts_tablet_manager.cc:1428] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee: Time spent starting tablet: real 0.008s user 0.001s sys 0.008s
I20260809 03:25:08.845124 10522 heartbeater.cc:503] Master 127.9.46.254:44289 was elected leader, sending a full tablet report...
I20260809 03:25:08.844942 10543 consensus_queue.cc:260] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0c618dedc44df897c90bd71f766472" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 46689 } } peers { permanent_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 35349 } } peers { permanent_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 46619 } }
I20260809 03:25:08.846756 10233 heartbeater.cc:503] Master 127.9.46.254:44289 was elected leader, sending a full tablet report...
I20260809 03:25:08.846976 10543 ts_tablet_manager.cc:1428] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
W20260809 03:25:09.010387 10523 tablet.cc:2369] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:25:09.039335 10379 tablet.cc:2369] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:25:09.059928 10234 tablet.cc:2369] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:09.062464 10549 raft_consensus.cc:482] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:09.062700 10549 raft_consensus.cc:504] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0c618dedc44df897c90bd71f766472" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 46689 } } peers { permanent_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 35349 } } peers { permanent_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 46619 } }
I20260809 03:25:09.064090 10549 leader_election.cc:288] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b36a411f680487cb32d0dabe5b8c1ee (127.9.46.194:35349), 8c9d8c6e4c2248ecac9f6eb8156770dd (127.9.46.193:46619)
I20260809 03:25:09.068744 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:09.064349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:46619 (local address 127.9.46.195:60149)
0809 03:25:09.064489 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:09.064505 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:09.064665 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:09.064991 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:09.064993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:09.065013 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:09.065213 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:09.065221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.066329 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.066332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:09.067253 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:09.067263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.068220 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.068224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:09.068322 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:09.068354 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 03:25:09.068486 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 03:25:09.068596 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 03:25:09.068827 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:09.064629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:60149 (local address 127.9.46.193:46619)
0809 03:25:09.064759 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:09.064762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:09.064776 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:25:09.064828 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:09.064830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:09.064868 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:09.064937 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:09.065351 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.066214 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.067416 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.068053 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.068182 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:09.068235 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:09.068529 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 03:25:09.068697 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:25:09.069618 10182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c513ccad67147efabc42b2490fab7df" candidate_uuid: "9a0c618dedc44df897c90bd71f766472" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" is_pre_election: true
I20260809 03:25:09.069977 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:09.064376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:35349 (local address 127.9.46.195:38665)
0809 03:25:09.065214 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:09.065225 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:09.065342 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:09.065552 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:09.065555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:09.065573 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:09.065849 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:09.065858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.066694 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.066697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:09.068298 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:09.068308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.069407 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.069412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:09.069564 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:09.069612 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 03:25:09.069708 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 03:25:09.069806 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":348,"thread_start_us":190,"threads_started":1}
I20260809 03:25:09.069970 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:09.064391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:38665 (local address 127.9.46.194:35349)
0809 03:25:09.064517 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:09.064523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:09.065354 (+ 831us) server_negotiation.cc:407] Connection header received
0809 03:25:09.065414 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:09.065416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:09.065459 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:09.065515 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:09.065971 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.066557 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.068451 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.069168 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.069271 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:09.069308 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:09.069734 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 03:25:09.069796 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:25:09.070099 10182 raft_consensus.cc:2454] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a0c618dedc44df897c90bd71f766472 in term 0.
I20260809 03:25:09.070662 10329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c513ccad67147efabc42b2490fab7df" candidate_uuid: "9a0c618dedc44df897c90bd71f766472" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" is_pre_election: true
I20260809 03:25:09.070796 10401 leader_election.cc:302] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c9d8c6e4c2248ecac9f6eb8156770dd, 9a0c618dedc44df897c90bd71f766472; no voters:
I20260809 03:25:09.070967 10329 raft_consensus.cc:2454] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a0c618dedc44df897c90bd71f766472 in term 0.
I20260809 03:25:09.071185 10549 raft_consensus.cc:2781] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:09.071281 10549 raft_consensus.cc:482] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:09.071358 10549 raft_consensus.cc:3037] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:09.072896 10549 raft_consensus.cc:504] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0c618dedc44df897c90bd71f766472" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 46689 } } peers { permanent_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 35349 } } peers { permanent_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 46619 } }
I20260809 03:25:09.073246 10549 leader_election.cc:288] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [CANDIDATE]: Term 1 election: Requested vote from peers 4b36a411f680487cb32d0dabe5b8c1ee (127.9.46.194:35349), 8c9d8c6e4c2248ecac9f6eb8156770dd (127.9.46.193:46619)
I20260809 03:25:09.073612 10329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c513ccad67147efabc42b2490fab7df" candidate_uuid: "9a0c618dedc44df897c90bd71f766472" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b36a411f680487cb32d0dabe5b8c1ee"
I20260809 03:25:09.073719 10329 raft_consensus.cc:3037] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:09.073747 10182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c513ccad67147efabc42b2490fab7df" candidate_uuid: "9a0c618dedc44df897c90bd71f766472" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd"
I20260809 03:25:09.073891 10182 raft_consensus.cc:3037] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:09.075340 10329 raft_consensus.cc:2454] T 2c513ccad67147efabc42b2490fab7df P 4b36a411f680487cb32d0dabe5b8c1ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a0c618dedc44df897c90bd71f766472 in term 1.
I20260809 03:25:09.075666 10182 raft_consensus.cc:2454] T 2c513ccad67147efabc42b2490fab7df P 8c9d8c6e4c2248ecac9f6eb8156770dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a0c618dedc44df897c90bd71f766472 in term 1.
I20260809 03:25:09.075809 10399 leader_election.cc:302] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b36a411f680487cb32d0dabe5b8c1ee, 9a0c618dedc44df897c90bd71f766472; no voters:
I20260809 03:25:09.076045 10549 raft_consensus.cc:2781] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:09.076346 10549 raft_consensus.cc:686] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [term 1 LEADER]: Becoming Leader. State: Replica: 9a0c618dedc44df897c90bd71f766472, State: Running, Role: LEADER
I20260809 03:25:09.076746 10549 consensus_queue.cc:237] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0c618dedc44df897c90bd71f766472" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 46689 } } peers { permanent_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 35349 } } peers { permanent_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 46619 } }
I20260809 03:25:09.080284 10019 catalog_manager.cc:5495] T 2c513ccad67147efabc42b2490fab7df P 9a0c618dedc44df897c90bd71f766472 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a0c618dedc44df897c90bd71f766472 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "9a0c618dedc44df897c90bd71f766472" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0c618dedc44df897c90bd71f766472" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 46689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b36a411f680487cb32d0dabe5b8c1ee" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 35349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c9d8c6e4c2248ecac9f6eb8156770dd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 46619 } health_report { overall_health: UNKNOWN } } }
Aug 09 03:25:09 dist-test-slave-65mx krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:25:09.108477 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:09.098295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35606 (local address 127.9.46.193:46619)
0809 03:25:09.098434 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:09.098439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:09.098744 (+ 305us) server_negotiation.cc:407] Connection header received
0809 03:25:09.098918 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:09.098920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:09.098965 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:09.099059 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:09.099866 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.100417 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.101616 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.101837 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.101952 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:09.104734 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:09.104767 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:09.104781 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:09.104808 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:09.106623 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:09.107111 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:09.107116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:09.107117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:09.107199 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:09.107488 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:09.107492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:09.107494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:09.107745 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:09.107963 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:09.108129 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 03:25:09.108259 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 03:25:09 dist-test-slave-65mx krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:25:09 dist-test-slave-65mx krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:25:09.138890 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:09.129902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.9.46.193:46619)
0809 03:25:09.130070 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:09.130073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:09.130120 (+ 47us) server_negotiation.cc:407] Connection header received
0809 03:25:09.130260 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:09.130263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:09.130305 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:09.130414 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:09.131147 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.131788 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.132673 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.132892 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.132973 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:09.135752 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:09.135754 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:09.135756 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:09.135781 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:09.137300 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:09.137733 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:09.137735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:09.137736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:09.137809 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:09.138056 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:09.138058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:09.138059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:09.138233 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:09.138332 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:09.138663 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 03:25:09.138763 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 03:25:09 dist-test-slave-65mx krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:25:09 dist-test-slave-65mx krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:25:09.168490 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:09.159510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35632 (local address 127.9.46.193:46619)
0809 03:25:09.159758 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:09.159766 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:09.160060 (+ 294us) server_negotiation.cc:407] Connection header received
0809 03:25:09.160255 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:09.160259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:09.160328 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:09.160419 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:09.161208 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.161732 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.162557 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:09.162769 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:09.162875 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:09.165437 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:09.165440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:09.165442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:09.165468 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:09.166961 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:09.167444 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:09.167446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:09.167447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:09.167520 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:09.167755 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:09.167757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:09.167758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:09.167918 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:09.168015 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:09.168261 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 03:25:09.168364 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 03:25:09.169920 10138 tablet_service.cc:2908] Rejecting scan request for tablet 2c513ccad67147efabc42b2490fab7df: Uninitialized: clean time has not yet been initialized
I20260809 03:25:09.172063 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 10091
I20260809 03:25:09.180478 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 10239
I20260809 03:25:09.187951 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 10383
I20260809 03:25:09.194734 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 9986
2026-08-09T03:25:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2581 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:09 dist-test-slave-65mx krb5kdc[10567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:09 dist-test-slave-65mx krb5kdc[10567](info): set up 2 sockets
Aug 09 03:25:09 dist-test-slave-65mx krb5kdc[10567](info): commencing operation
krb5kdc: starting...
W20260809 03:25:10.269214 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:10 dist-test-slave-65mx krb5kdc[10567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:10.395912 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34099
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:34099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:10.506361 10582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:10.506671 10582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:10.506752 10582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:10.510844 10582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:10.510932 10582 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 03:25:10.510962 10582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:10.511025 10582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:10.511081 10582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:10.511124 10582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:10.516480 10582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:34099
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:10.517668 10582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:10.518697 10582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:10.527295 10590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:10.527302 10587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:10.527370 10582 server_base.cc:1034] running on GCE node
W20260809 03:25:10.527302 10588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:10.528223 10582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:10.529358 10582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:10.530560 10582 hybrid_clock.cc:648] HybridClock initialized: now 1786245910530527 us; error 46 us; skew 500 ppm
Aug 09 03:25:10 dist-test-slave-65mx krb5kdc[10567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245910, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:10.533417 10582 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:10.534334 10582 webserver.cc:460] Webserver started at http://127.9.46.254:45953/ using document root <none> and password file <none>
I20260809 03:25:10.534889 10582 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:10.534982 10582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:10.535225 10582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:10.537295 10582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "bfa804c3390048a78947be90c1bb54f0"
format_stamp: "Formatted at 2026-08-09 03:25:10 on dist-test-slave-65mx"
server_key: "0204459066365b7293eb44ba41a48601"
server_key_iv: "07f49eff4bbbab7c41238621f1ae3e8b"
server_key_version: "encryptionkey@0"
I20260809 03:25:10.537843 10582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "bfa804c3390048a78947be90c1bb54f0"
format_stamp: "Formatted at 2026-08-09 03:25:10 on dist-test-slave-65mx"
server_key: "0204459066365b7293eb44ba41a48601"
server_key_iv: "07f49eff4bbbab7c41238621f1ae3e8b"
server_key_version: "encryptionkey@0"
I20260809 03:25:10.541474 10582 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:25:10.543792 10597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:10.545063 10582 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:10.545212 10582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "bfa804c3390048a78947be90c1bb54f0"
format_stamp: "Formatted at 2026-08-09 03:25:10 on dist-test-slave-65mx"
server_key: "0204459066365b7293eb44ba41a48601"
server_key_iv: "07f49eff4bbbab7c41238621f1ae3e8b"
server_key_version: "encryptionkey@0"
I20260809 03:25:10.545322 10582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:10.563109 10582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:10.566550 10582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:10.566782 10582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:10.575350 10582 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:34099
I20260809 03:25:10.575377 10664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:34099 every 8 connection(s)
I20260809 03:25:10.576341 10582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:10.579074 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:10.582610 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 10582
I20260809 03:25:10.582731 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:10.583056 9403 external_mini_cluster.cc:1428] Setting key 282e6fba4c1c7158b9c16e906b8eac2b
I20260809 03:25:10.585238 10665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0: Bootstrap starting.
I20260809 03:25:10.587493 10665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:10.588244 10665 log.cc:824] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:10.590169 10665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0: No bootstrap required, opened a new log
Aug 09 03:25:10 dist-test-slave-65mx krb5kdc[10567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:10.594074 10665 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa804c3390048a78947be90c1bb54f0" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34099 } }
I20260809 03:25:10.594304 10665 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:10.594381 10665 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa804c3390048a78947be90c1bb54f0, State: Initialized, Role: FOLLOWER
I20260809 03:25:10.594803 10665 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa804c3390048a78947be90c1bb54f0" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34099 } }
I20260809 03:25:10.594924 10665 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:10.595009 10665 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:10.595150 10665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:10.596027 10665 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa804c3390048a78947be90c1bb54f0" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34099 } }
I20260809 03:25:10.596365 10665 leader_election.cc:302] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfa804c3390048a78947be90c1bb54f0; no voters:
I20260809 03:25:10.596750 10665 leader_election.cc:288] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:10.596853 10671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:10.597092 10671 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [term 1 LEADER]: Becoming Leader. State: Replica: bfa804c3390048a78947be90c1bb54f0, State: Running, Role: LEADER
I20260809 03:25:10.597411 10671 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa804c3390048a78947be90c1bb54f0" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34099 } }
I20260809 03:25:10.598104 10665 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:10.598424 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:10.584594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.9.46.254:34099)
0809 03:25:10.585033 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:10.585049 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:10.585085 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:25:10.585784 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:10.585802 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:10.586128 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:10.586469 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:10.587293 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:10.588347 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:10.589444 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:10.589914 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:10.590088 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:10.592872 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:10.592877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:10.592890 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:10.592919 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:10.594911 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:10.595470 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:10.595476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:10.595481 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:10.595608 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:10.595890 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:10.595895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:10.595896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:10.596365 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:10.596595 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:10.597057 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 03:25:10.597422 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":137,"threads_started":1}
I20260809 03:25:10.598519 10671 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa804c3390048a78947be90c1bb54f0. Latest consensus state: current_term: 1 leader_uuid: "bfa804c3390048a78947be90c1bb54f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa804c3390048a78947be90c1bb54f0" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34099 } } }
I20260809 03:25:10.598641 10671 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:10.598941 10673 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfa804c3390048a78947be90c1bb54f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa804c3390048a78947be90c1bb54f0" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34099 } } }
I20260809 03:25:10.599026 10673 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:10.600621 10675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:10.603343 10675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:10.609349 10675 catalog_manager.cc:1347] Generated new cluster ID: 96977be133c74ad0a9994d76d212554e
I20260809 03:25:10.609431 10675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:10.621624 10675 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:10.622630 10675 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:10.627869 10675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0: Generated new TSK 0
I20260809 03:25:10.628659 10675 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:25:10.664013 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34099
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:10.782785 10694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:10.783052 10694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:10.783097 10694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:10.787289 10694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:10.787354 10694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:10.787397 10694 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:25:10.787431 10694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:10.792865 10694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.46.254:34099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:10.793860 10694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:10.794838 10694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:10.804821 10703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:10.804821 10699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:10.804824 10701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:10.805122 10694 server_base.cc:1034] running on GCE node
I20260809 03:25:10.805667 10694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:10.806303 10694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:10.807507 10694 hybrid_clock.cc:648] HybridClock initialized: now 1786245910807492 us; error 28 us; skew 500 ppm
Aug 09 03:25:10 dist-test-slave-65mx krb5kdc[10567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245910, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:10.810549 10694 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:25:10.811403 10694 webserver.cc:460] Webserver started at http://127.9.46.193:42965/ using document root <none> and password file <none>
I20260809 03:25:10.811928 10694 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:10.811985 10694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:10.812162 10694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:10.814242 10694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "df8ea0036fb44764b6a8ce36f6934e16"
format_stamp: "Formatted at 2026-08-09 03:25:10 on dist-test-slave-65mx"
server_key: "4a73e39e649e8abfc90fee61696a87a1"
server_key_iv: "5e987a10ad2324a652836567ff9cdd70"
server_key_version: "encryptionkey@0"
I20260809 03:25:10.814759 10694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "df8ea0036fb44764b6a8ce36f6934e16"
format_stamp: "Formatted at 2026-08-09 03:25:10 on dist-test-slave-65mx"
server_key: "4a73e39e649e8abfc90fee61696a87a1"
server_key_iv: "5e987a10ad2324a652836567ff9cdd70"
server_key_version: "encryptionkey@0"
I20260809 03:25:10.818701 10694 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:25:10.821132 10710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:10.822330 10694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:10.822453 10694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "df8ea0036fb44764b6a8ce36f6934e16"
format_stamp: "Formatted at 2026-08-09 03:25:10 on dist-test-slave-65mx"
server_key: "4a73e39e649e8abfc90fee61696a87a1"
server_key_iv: "5e987a10ad2324a652836567ff9cdd70"
server_key_version: "encryptionkey@0"
I20260809 03:25:10.822563 10694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:10.836755 10694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:10.839622 10694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:10.839828 10694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:10.841375 10694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:10.841424 10694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:10.841464 10694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:10.841519 10694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:10.852838 10694 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:35185
Aug 09 03:25:10 dist-test-slave-65mx krb5kdc[10567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245910, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:10.852910 10830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:35185 every 8 connection(s)
I20260809 03:25:10.853720 10694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:10.857530 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:10.842061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:47357 (local address 127.9.46.254:34099)
0809 03:25:10.842192 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:10.842196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:10.843838 (+ 1642us) server_negotiation.cc:407] Connection header received
0809 03:25:10.845147 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:10.845152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:10.845215 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:10.845276 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:10.847059 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:10.847956 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:10.849908 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:10.850176 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:10.850274 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:10.853690 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:10.853695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:10.853698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:10.853738 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:10.855423 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:10.855931 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:10.855933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:10.855935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:10.856012 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:10.856412 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:10.856414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:10.856415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:10.856685 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:10.856766 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:10.857289 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 03:25:10.857399 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:25:10.858366 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:10.843126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34099 (local address 127.9.46.193:47357)
0809 03:25:10.843674 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:10.843731 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:10.844884 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:10.845704 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:10.845712 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:10.846193 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:10.846868 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:10.846891 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:10.848127 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:10.848130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:10.849749 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:10.849760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:10.850356 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:10.850367 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:10.850612 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:10.851427 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:10.851451 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:10.853463 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:10.855538 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:10.855543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:10.855554 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:10.855829 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:10.856140 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:10.856147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:10.856149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:10.856309 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:10.856787 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:10.856792 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:10.857045 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 03:25:10.857223 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 03:25:10.857481 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":134,"threads_started":1}
I20260809 03:25:10.859565 10831 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34099
I20260809 03:25:10.859727 10831 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:10.860271 10831 heartbeater.cc:511] Master 127.9.46.254:34099 requested a full tablet report, sending...
I20260809 03:25:10.861929 10614 ts_manager.cc:194] Registered new tserver with Master: df8ea0036fb44764b6a8ce36f6934e16 (127.9.46.193:35185)
I20260809 03:25:10.862232 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 10694
I20260809 03:25:10.862355 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:10.862705 9403 external_mini_cluster.cc:1428] Setting key 6059c9b44eb4a095e325c44b4340ad8b
I20260809 03:25:10.863296 10614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:47357
I20260809 03:25:10.870762 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:10.863109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:58877 (local address 127.9.46.254:34099)
0809 03:25:10.863226 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:10.863230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:10.863245 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:25:10.863281 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:10.863284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:10.863327 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:10.863410 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:10.863894 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:10.864675 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:10.865967 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:10.866211 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:10.866337 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:10.867488 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:10.867496 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:10.867498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:10.867520 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:10.869026 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:10.869506 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:10.869511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:10.869513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:10.869599 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:10.869900 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:10.869903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:10.869905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:10.870092 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:10.870208 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:10.870480 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 03:25:10.870598 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:25:10.870776 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:10.861924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34099 (local address 127.9.46.193:58877)
0809 03:25:10.862309 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:10.862326 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:10.862417 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:10.863426 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:10.863429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:10.863636 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:10.863736 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:10.863741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:10.864849 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:10.864855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:10.865837 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:10.865852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:10.866341 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:10.866344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:10.866440 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:10.866994 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:10.867006 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:10.867343 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:10.869178 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:10.869182 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:10.869184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:10.869400 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:10.869700 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:10.869705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:10.869707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:10.869798 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:10.870197 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:10.870200 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:10.870305 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 03:25:10.870423 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 03:25:10.870586 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:25:10.892235 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34099
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:11.006680 10840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:11.006965 10840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:11.007051 10840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:11.011325 10840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:11.011409 10840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:11.011477 10840 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:25:11.011562 10840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:11.016882 10840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.46.254:34099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:11.017937 10840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:11.018998 10840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:11.028445 10848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:11.028479 10846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:11.028636 10845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:11.028956 10840 server_base.cc:1034] running on GCE node
I20260809 03:25:11.029345 10840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:11.029958 10840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:11.031139 10840 hybrid_clock.cc:648] HybridClock initialized: now 1786245911031103 us; error 48 us; skew 500 ppm
Aug 09 03:25:11 dist-test-slave-65mx krb5kdc[10567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245911, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:11.033895 10840 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:25:11.034789 10840 webserver.cc:460] Webserver started at http://127.9.46.194:36973/ using document root <none> and password file <none>
I20260809 03:25:11.035286 10840 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:11.035344 10840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:11.035501 10840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:11.037503 10840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "1f91f5c79c024b60b91028e9b588cf39"
format_stamp: "Formatted at 2026-08-09 03:25:11 on dist-test-slave-65mx"
server_key: "b5b3c28827bbef38ee173929e6fb100b"
server_key_iv: "3327645b449ea1f405a5fc92f7cca176"
server_key_version: "encryptionkey@0"
I20260809 03:25:11.038019 10840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "1f91f5c79c024b60b91028e9b588cf39"
format_stamp: "Formatted at 2026-08-09 03:25:11 on dist-test-slave-65mx"
server_key: "b5b3c28827bbef38ee173929e6fb100b"
server_key_iv: "3327645b449ea1f405a5fc92f7cca176"
server_key_version: "encryptionkey@0"
I20260809 03:25:11.041579 10840 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:25:11.043800 10855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:11.045027 10840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:11.045159 10840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "1f91f5c79c024b60b91028e9b588cf39"
format_stamp: "Formatted at 2026-08-09 03:25:11 on dist-test-slave-65mx"
server_key: "b5b3c28827bbef38ee173929e6fb100b"
server_key_iv: "3327645b449ea1f405a5fc92f7cca176"
server_key_version: "encryptionkey@0"
I20260809 03:25:11.045276 10840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:11.074352 10840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:11.077220 10840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:11.077430 10840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:11.078955 10840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:11.079036 10840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:11.079104 10840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:11.079145 10840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:25:11 dist-test-slave-65mx krb5kdc[10567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245911, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:11.090272 10840 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:45169
I20260809 03:25:11.090408 10977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:45169 every 8 connection(s)
I20260809 03:25:11.091398 10840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:11.093624 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.079712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:48133 (local address 127.9.46.254:34099)
0809 03:25:11.079894 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.079901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.081210 (+ 1309us) server_negotiation.cc:407] Connection header received
0809 03:25:11.082324 (+ 1114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.082331 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.082401 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.082494 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:11.083868 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.084462 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.086396 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.086708 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.086832 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.089601 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:11.089606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:11.089612 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:11.089648 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:11.091444 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.092020 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.092024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.092025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.092119 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.092480 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.092482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.092483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.092748 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:11.092833 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.093398 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.093502 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 03:25:11.094445 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.080504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34099 (local address 127.9.46.194:48133)
0809 03:25:11.081029 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.081090 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.082104 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.082684 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.082696 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.083066 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:11.083726 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.083740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.084642 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.084648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.086250 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.086265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.086877 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.086881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.087074 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.087777 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:11.087793 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:11.089423 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:11.091604 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:11.091608 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:11.091619 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:11.091892 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:11.092228 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:11.092231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:11.092233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:11.092374 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:11.092907 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:11.092912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:11.093176 (+ 264us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.093340 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.093585 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":132,"threads_started":1}
I20260809 03:25:11.095728 10978 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34099
I20260809 03:25:11.095873 10978 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:11.096406 10978 heartbeater.cc:511] Master 127.9.46.254:34099 requested a full tablet report, sending...
I20260809 03:25:11.097675 10614 ts_manager.cc:194] Registered new tserver with Master: 1f91f5c79c024b60b91028e9b588cf39 (127.9.46.194:45169)
I20260809 03:25:11.098412 10614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:48133
I20260809 03:25:11.099624 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 10840
I20260809 03:25:11.099737 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:11.099993 9403 external_mini_cluster.cc:1428] Setting key 9f99e8a20d91c512c43d1303ccd13a21
I20260809 03:25:11.105957 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.098201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:49739 (local address 127.9.46.254:34099)
0809 03:25:11.098319 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.098323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.098522 (+ 199us) server_negotiation.cc:407] Connection header received
0809 03:25:11.098634 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.098636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.098704 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.098782 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:11.099330 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.100082 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.101100 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.101391 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.101501 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.102633 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:11.102639 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:11.102646 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:11.102679 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:11.104166 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.104757 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.104761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.104763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.104848 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.105156 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.105158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.105159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.105354 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:11.105431 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.105755 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.105848 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:25:11.106202 10984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.098120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34099 (local address 127.9.46.194:49739)
0809 03:25:11.098419 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.098437 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.098539 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.098817 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.098820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.099057 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:11.099207 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.099215 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.100253 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.100256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.101008 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.101018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.101511 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.101514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.101585 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.102160 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:11.102171 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:11.102490 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:11.104280 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:11.104286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:11.104291 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:11.104646 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:11.104947 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:11.104954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:11.104959 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:11.105062 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:11.105452 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:11.105458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:11.105576 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.105706 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.105876 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:25:11.131727 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34099
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:11.248608 10987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:11.248884 10987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:11.248967 10987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:11.253082 10987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:11.253144 10987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:11.253182 10987 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:25:11.253214 10987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:11.258833 10987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.46.254:34099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:11.259881 10987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:11.260910 10987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:11.270356 10992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:11.270362 10993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:11.270335 10995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:11.270722 10987 server_base.cc:1034] running on GCE node
I20260809 03:25:11.271168 10987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:11.271816 10987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:11.273036 10987 hybrid_clock.cc:648] HybridClock initialized: now 1786245911273017 us; error 29 us; skew 500 ppm
Aug 09 03:25:11 dist-test-slave-65mx krb5kdc[10567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245911, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:11.275854 10987 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:25:11.276883 10987 webserver.cc:460] Webserver started at http://127.9.46.195:34751/ using document root <none> and password file <none>
I20260809 03:25:11.277458 10987 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:11.277555 10987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:11.277793 10987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:11.279800 10987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "d1c4a490361441d5ad05d3544fad15a5"
format_stamp: "Formatted at 2026-08-09 03:25:11 on dist-test-slave-65mx"
server_key: "1dfe0f83f5429c5817c379a3213f9c29"
server_key_iv: "fb5b491dd2e53295d174dab59d5bdcca"
server_key_version: "encryptionkey@0"
I20260809 03:25:11.280362 10987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "d1c4a490361441d5ad05d3544fad15a5"
format_stamp: "Formatted at 2026-08-09 03:25:11 on dist-test-slave-65mx"
server_key: "1dfe0f83f5429c5817c379a3213f9c29"
server_key_iv: "fb5b491dd2e53295d174dab59d5bdcca"
server_key_version: "encryptionkey@0"
I20260809 03:25:11.283946 10987 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:25:11.286520 11002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:11.287812 10987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:11.287971 10987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "d1c4a490361441d5ad05d3544fad15a5"
format_stamp: "Formatted at 2026-08-09 03:25:11 on dist-test-slave-65mx"
server_key: "1dfe0f83f5429c5817c379a3213f9c29"
server_key_iv: "fb5b491dd2e53295d174dab59d5bdcca"
server_key_version: "encryptionkey@0"
I20260809 03:25:11.288092 10987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:11.304060 10987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:11.306912 10987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:11.307123 10987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:11.308686 10987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:11.308758 10987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:11.308830 10987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:11.308887 10987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:25:11 dist-test-slave-65mx krb5kdc[10567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245911, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:11.320690 10987 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:43647
I20260809 03:25:11.321772 10987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:11.323415 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.309407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:45987 (local address 127.9.46.254:34099)
0809 03:25:11.309540 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.309544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.311245 (+ 1701us) server_negotiation.cc:407] Connection header received
0809 03:25:11.312383 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.312387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.312449 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.312531 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:11.313994 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.314642 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.315825 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.316231 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.316382 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.319488 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:11.319492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:11.319494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:11.319523 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:11.321217 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.321708 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.321713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.321715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.321818 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.322236 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.322240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.322241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.322465 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:11.322566 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.323187 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.323294 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:25:11.322628 11124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:43647 every 8 connection(s)
I20260809 03:25:11.324172 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.310545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34099 (local address 127.9.46.195:45987)
0809 03:25:11.311054 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.311131 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.312175 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.312774 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.312784 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.313302 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:11.313847 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.313862 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.314772 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.314777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.315679 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.315688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.316430 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.316438 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.316681 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.317291 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:11.317307 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:11.319265 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:11.321343 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:11.321348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:11.321357 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:11.321599 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:11.321939 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:11.321942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:11.321945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:11.322119 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:11.322659 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:11.322665 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:11.322921 (+ 256us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.323075 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.323329 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":161,"threads_started":1}
I20260809 03:25:11.325501 11125 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34099
I20260809 03:25:11.325666 11125 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:11.326161 11125 heartbeater.cc:511] Master 127.9.46.254:34099 requested a full tablet report, sending...
I20260809 03:25:11.327296 10615 ts_manager.cc:194] Registered new tserver with Master: d1c4a490361441d5ad05d3544fad15a5 (127.9.46.195:43647)
I20260809 03:25:11.328125 10615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:45987
I20260809 03:25:11.328812 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 10987
I20260809 03:25:11.328948 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:11.329216 9403 external_mini_cluster.cc:1428] Setting key 37d425a9df68b6723de953890b15b603
I20260809 03:25:11.331215 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:25:11.335726 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.327763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:44511 (local address 127.9.46.254:34099)
0809 03:25:11.327877 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.327881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.328086 (+ 205us) server_negotiation.cc:407] Connection header received
0809 03:25:11.328192 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.328194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.328261 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.328339 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:11.328878 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.329609 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.330657 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.330975 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.331080 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.332320 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:11.332323 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:11.332325 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:11.332347 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:11.334082 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.334487 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.334494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.334499 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.334611 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.334872 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.334878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.334882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.335070 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:11.335199 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.335506 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.335614 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:25:11.335907 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.327684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34099 (local address 127.9.46.195:44511)
0809 03:25:11.327995 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.328008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.328110 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.328365 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.328369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.328611 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:11.328748 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.328756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.329763 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.329766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.330543 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.330552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.331108 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.331110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.331223 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.331697 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:11.331713 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:11.332183 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:11.334209 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:11.334211 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:11.334213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:11.334396 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:11.334704 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:11.334706 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:11.334707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:11.334781 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:11.335213 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:11.335219 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:11.335331 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.335451 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.335630 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":92,"threads_started":1}
Aug 09 03:25:11 dist-test-slave-65mx krb5kdc[10567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245911, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:25:11 dist-test-slave-65mx krb5kdc[10567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245911, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:11.359347 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.351434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34632 (local address 127.9.46.254:34099)
0809 03:25:11.351574 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.351578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.351590 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:25:11.351630 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.351632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.351669 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.351761 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:11.352465 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.353036 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.353883 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.354090 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.354201 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.356605 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:11.356608 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:11.356611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:11.356634 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:11.357874 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.358269 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.358272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.358273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.358348 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.358582 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.358584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.358585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.358748 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:11.358846 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.359130 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.359224 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:25:11.362471 10615 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:11.365465 10615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:11.380311 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.373875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.9.46.194:45169)
0809 03:25:11.375230 (+ 1355us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.375243 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.375261 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:25:11.375324 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.375328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.375467 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.375628 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:11.376405 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.377498 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.378771 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.379617 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.379734 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.379900 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.380063 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.380157 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1255,"thread_start_us":100,"threads_started":1}
I20260809 03:25:11.380321 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.373709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:45169 (local address 127.0.0.1:56244)
0809 03:25:11.374341 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.374392 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.374527 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.375641 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.375647 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.375681 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:11.375937 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.375945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.377636 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.377639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.378635 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.378645 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.379752 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.379755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.379871 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.379911 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.380026 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.380142 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":86,"spinlock_wait_cycles":13440,"thread_start_us":89,"threads_started":1}
I20260809 03:25:11.380815 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.373810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:43647 (local address 127.0.0.1:57784)
0809 03:25:11.374356 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.374392 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.374527 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.376221 (+ 1694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.376223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.376239 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:11.376311 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.376315 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.377457 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.377461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.379335 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.379344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.380328 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.380335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.380444 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.380479 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.380607 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.380685 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":90,"spinlock_wait_cycles":5120,"thread_start_us":105,"threads_started":1}
I20260809 03:25:11.381107 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.374680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.9.46.193:35185)
0809 03:25:11.375147 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.375155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.375180 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:25:11.375262 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.375267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.375466 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.375628 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:11.376075 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.377290 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.378821 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.379624 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.379735 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.379882 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.380884 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.380982 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":87,"threads_started":1}
I20260809 03:25:11.381129 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.374798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57784 (local address 127.9.46.195:43647)
0809 03:25:11.375116 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.375129 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.375156 (+ 27us) server_negotiation.cc:407] Connection header received
0809 03:25:11.375914 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.375918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.376088 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.376238 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:11.376433 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.377317 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.379456 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.380201 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.380303 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.380461 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.380899 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.380981 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":89,"threads_started":1}
I20260809 03:25:11.381029 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.374088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:35185 (local address 127.0.0.1:56338)
0809 03:25:11.374765 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.374778 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.374857 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.375641 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.375647 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.375679 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:11.375937 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.375945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.377422 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.377425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.378710 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.378722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.380493 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.380497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.380607 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.380641 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.380715 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.380789 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":218,"spinlock_wait_cycles":896,"thread_start_us":100,"threads_started":1}
I20260809 03:25:11.383757 10762 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9ecd9a63a847f0b8ea81bd056fa091 (DEFAULT_TABLE table=test-table [id=06eac51dd66c43ba9429228d18fd5602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:11.383909 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9ecd9a63a847f0b8ea81bd056fa091 (DEFAULT_TABLE table=test-table [id=06eac51dd66c43ba9429228d18fd5602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:11.383913 11054 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9ecd9a63a847f0b8ea81bd056fa091 (DEFAULT_TABLE table=test-table [id=06eac51dd66c43ba9429228d18fd5602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:11.384862 10762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9ecd9a63a847f0b8ea81bd056fa091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:11.384928 11054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9ecd9a63a847f0b8ea81bd056fa091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:11.384923 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9ecd9a63a847f0b8ea81bd056fa091. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:11.390805 11153 tablet_bootstrap.cc:492] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: Bootstrap starting.
I20260809 03:25:11.392735 11154 tablet_bootstrap.cc:492] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: Bootstrap starting.
I20260809 03:25:11.393185 11153 tablet_bootstrap.cc:654] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:11.393743 11155 tablet_bootstrap.cc:492] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: Bootstrap starting.
I20260809 03:25:11.394131 11153 log.cc:824] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:11.395100 11154 tablet_bootstrap.cc:654] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:11.395875 11155 tablet_bootstrap.cc:654] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:11.395958 11154 log.cc:824] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:11.396476 11153 tablet_bootstrap.cc:492] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: No bootstrap required, opened a new log
I20260809 03:25:11.396739 11153 ts_tablet_manager.cc:1397] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.002s
I20260809 03:25:11.396713 11155 log.cc:824] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:11.398219 11154 tablet_bootstrap.cc:492] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: No bootstrap required, opened a new log
I20260809 03:25:11.398434 11154 ts_tablet_manager.cc:1397] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 03:25:11.398455 11155 tablet_bootstrap.cc:492] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: No bootstrap required, opened a new log
I20260809 03:25:11.398638 11155 ts_tablet_manager.cc:1397] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 03:25:11.401567 11153 raft_consensus.cc:348] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } }
I20260809 03:25:11.401655 11155 raft_consensus.cc:348] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } }
I20260809 03:25:11.401839 11153 raft_consensus.cc:374] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:11.401873 11155 raft_consensus.cc:374] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:11.401891 11153 raft_consensus.cc:729] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df8ea0036fb44764b6a8ce36f6934e16, State: Initialized, Role: FOLLOWER
I20260809 03:25:11.401929 11155 raft_consensus.cc:729] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f91f5c79c024b60b91028e9b588cf39, State: Initialized, Role: FOLLOWER
I20260809 03:25:11.402194 11154 raft_consensus.cc:348] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } }
I20260809 03:25:11.402405 11154 raft_consensus.cc:374] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:11.402446 11154 raft_consensus.cc:729] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c4a490361441d5ad05d3544fad15a5, State: Initialized, Role: FOLLOWER
I20260809 03:25:11.402376 11153 consensus_queue.cc:260] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } }
I20260809 03:25:11.402424 11155 consensus_queue.cc:260] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } }
I20260809 03:25:11.402841 11154 consensus_queue.cc:260] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } }
I20260809 03:25:11.403380 11153 ts_tablet_manager.cc:1428] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260809 03:25:11.403385 11155 ts_tablet_manager.cc:1428] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:25:11.403543 11125 heartbeater.cc:503] Master 127.9.46.254:34099 was elected leader, sending a full tablet report...
I20260809 03:25:11.403672 10978 heartbeater.cc:503] Master 127.9.46.254:34099 was elected leader, sending a full tablet report...
I20260809 03:25:11.403726 11154 ts_tablet_manager.cc:1428] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:25:11.403822 10831 heartbeater.cc:503] Master 127.9.46.254:34099 was elected leader, sending a full tablet report...
I20260809 03:25:11.552412 11161 raft_consensus.cc:482] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:11.552781 11161 raft_consensus.cc:504] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } }
I20260809 03:25:11.554018 11161 leader_election.cc:288] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f91f5c79c024b60b91028e9b588cf39 (127.9.46.194:45169), df8ea0036fb44764b6a8ce36f6934e16 (127.9.46.193:35185)
I20260809 03:25:11.557699 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.554347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:34203 (local address 127.9.46.194:45169)
0809 03:25:11.554514 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.554519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.554535 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:25:11.554675 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.554678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.554732 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.554816 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:11.555089 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.555607 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.556710 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.557141 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.557276 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.557333 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.557505 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.557586 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 03:25:11.557699 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.554268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:45169 (local address 127.9.46.195:34203)
0809 03:25:11.554409 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.554427 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.554531 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.554830 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.554833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.554849 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:11.554972 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.554977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.555718 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.555738 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.556532 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.556593 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.557266 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.557269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.557370 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.557399 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.557488 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.557563 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 03:25:11.558378 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9ecd9a63a847f0b8ea81bd056fa091" candidate_uuid: "d1c4a490361441d5ad05d3544fad15a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f91f5c79c024b60b91028e9b588cf39" is_pre_election: true
I20260809 03:25:11.558691 10929 raft_consensus.cc:2454] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c4a490361441d5ad05d3544fad15a5 in term 0.
I20260809 03:25:11.558827 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.554776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:52759 (local address 127.9.46.193:35185)
0809 03:25:11.554919 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.554923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.554936 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:25:11.554996 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.554998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.555036 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.555123 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:11.555403 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.555938 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.557600 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.558223 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.558364 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.558412 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.558645 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.558723 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:25:11.558841 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.554338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:35185 (local address 127.9.46.195:52759)
0809 03:25:11.554691 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.554708 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.554816 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.555141 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.555144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.555160 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:11.555288 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.555293 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.556117 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.556120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.557441 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.557458 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.558367 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.558371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.558499 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.558531 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.558629 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.558697 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":159,"threads_started":1}
I20260809 03:25:11.559224 11004 leader_election.cc:302] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f91f5c79c024b60b91028e9b588cf39, d1c4a490361441d5ad05d3544fad15a5; no voters:
I20260809 03:25:11.559535 11161 raft_consensus.cc:2781] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:11.559621 11161 raft_consensus.cc:482] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:11.559561 10782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9ecd9a63a847f0b8ea81bd056fa091" candidate_uuid: "d1c4a490361441d5ad05d3544fad15a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df8ea0036fb44764b6a8ce36f6934e16" is_pre_election: true
I20260809 03:25:11.559687 11161 raft_consensus.cc:3037] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:11.559855 10782 raft_consensus.cc:2454] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c4a490361441d5ad05d3544fad15a5 in term 0.
I20260809 03:25:11.561020 11161 raft_consensus.cc:504] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } }
I20260809 03:25:11.561414 11161 leader_election.cc:288] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [CANDIDATE]: Term 1 election: Requested vote from peers 1f91f5c79c024b60b91028e9b588cf39 (127.9.46.194:45169), df8ea0036fb44764b6a8ce36f6934e16 (127.9.46.193:35185)
I20260809 03:25:11.561770 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9ecd9a63a847f0b8ea81bd056fa091" candidate_uuid: "d1c4a490361441d5ad05d3544fad15a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f91f5c79c024b60b91028e9b588cf39"
I20260809 03:25:11.561861 10782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9ecd9a63a847f0b8ea81bd056fa091" candidate_uuid: "d1c4a490361441d5ad05d3544fad15a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df8ea0036fb44764b6a8ce36f6934e16"
I20260809 03:25:11.561928 10929 raft_consensus.cc:3037] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:11.562011 10782 raft_consensus.cc:3037] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:11.563221 10929 raft_consensus.cc:2454] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c4a490361441d5ad05d3544fad15a5 in term 1.
I20260809 03:25:11.563416 10782 raft_consensus.cc:2454] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c4a490361441d5ad05d3544fad15a5 in term 1.
I20260809 03:25:11.563519 11004 leader_election.cc:302] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f91f5c79c024b60b91028e9b588cf39, d1c4a490361441d5ad05d3544fad15a5; no voters:
I20260809 03:25:11.563710 11161 raft_consensus.cc:2781] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:11.563980 11161 raft_consensus.cc:686] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 1 LEADER]: Becoming Leader. State: Replica: d1c4a490361441d5ad05d3544fad15a5, State: Running, Role: LEADER
I20260809 03:25:11.564301 11161 consensus_queue.cc:237] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } }
I20260809 03:25:11.567426 10614 catalog_manager.cc:5495] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1c4a490361441d5ad05d3544fad15a5 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "d1c4a490361441d5ad05d3544fad15a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } health_report { overall_health: UNKNOWN } } }
W20260809 03:25:11.572839 11127 tablet.cc:2369] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:25:11.592737 10979 tablet.cc:2369] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:25:11.605198 10832 tablet.cc:2369] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:11.623252 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.619009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57788 (local address 127.9.46.195:43647)
0809 03:25:11.619176 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.619182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.619200 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:25:11.619447 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.619450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.619519 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.619628 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:11.619996 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.620524 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.621503 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.621731 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.621865 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.622329 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:11.622768 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:11.622902 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.623031 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.623100 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:25:11.629786 10782 raft_consensus.cc:1261] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16 [term 1 FOLLOWER]: Refusing update from remote peer d1c4a490361441d5ad05d3544fad15a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:11.629799 10929 raft_consensus.cc:1261] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39 [term 1 FOLLOWER]: Refusing update from remote peer d1c4a490361441d5ad05d3544fad15a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:11.630476 11163 consensus_queue.cc:1035] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:11.630652 11161 consensus_queue.cc:1035] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:11.639405 11168 mvcc.cc:205] Tried to move back new op lower bound from 7316463254028640256 to 7316463253770424320. Current Snapshot: MvccSnapshot[applied={T|T < 7316463254028640256 or (T in {7316463254028640256})}]
I20260809 03:25:11.660336 10615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:11.660722 10615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:11.665115 10615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0: Sending DeleteTablet for 3 replicas of tablet 2c9ecd9a63a847f0b8ea81bd056fa091
I20260809 03:25:11.666317 11054 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c9ecd9a63a847f0b8ea81bd056fa091 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:11 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:57784
I20260809 03:25:11.667116 11176 tablet_replica.cc:330] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: stopping tablet replica
I20260809 03:25:11.667124 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c9ecd9a63a847f0b8ea81bd056fa091 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:11 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:56244
I20260809 03:25:11.667534 10762 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c9ecd9a63a847f0b8ea81bd056fa091 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:11 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:56338
I20260809 03:25:11.667686 11176 raft_consensus.cc:2229] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:11.668083 11176 raft_consensus.cc:2258] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:11.669696 10615 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:11.670399 10615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:11.670763 11176 ts_tablet_manager.cc:1905] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:11.676609 10762 tablet_service.cc:1467] Processing CreateTablet for tablet b6786d40bcb8475b93f991b1e64b4b12 (DEFAULT_TABLE table=test-table [id=9c4946f46ac540c0bf49b5ad229da25f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:11.676939 10762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6786d40bcb8475b93f991b1e64b4b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:11.676901 10909 tablet_service.cc:1467] Processing CreateTablet for tablet b6786d40bcb8475b93f991b1e64b4b12 (DEFAULT_TABLE table=test-table [id=9c4946f46ac540c0bf49b5ad229da25f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:11.677202 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6786d40bcb8475b93f991b1e64b4b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:11.680217 11153 tablet_bootstrap.cc:492] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16: Bootstrap starting.
I20260809 03:25:11.680505 11177 tablet_replica.cc:330] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: stopping tablet replica
I20260809 03:25:11.680918 11177 raft_consensus.cc:2229] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:11.681270 11177 raft_consensus.cc:2258] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:11.681301 11054 tablet_service.cc:1467] Processing CreateTablet for tablet b6786d40bcb8475b93f991b1e64b4b12 (DEFAULT_TABLE table=test-table [id=9c4946f46ac540c0bf49b5ad229da25f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:11.681563 11054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6786d40bcb8475b93f991b1e64b4b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:11.682551 11153 tablet_bootstrap.cc:654] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:11.684051 11155 tablet_bootstrap.cc:492] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39: Bootstrap starting.
I20260809 03:25:11.684881 11154 tablet_bootstrap.cc:492] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5: Bootstrap starting.
I20260809 03:25:11.686142 11177 ts_tablet_manager.cc:1905] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:11.686321 11155 tablet_bootstrap.cc:654] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:11.686672 11154 tablet_bootstrap.cc:654] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:11.689014 11178 tablet_replica.cc:330] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: stopping tablet replica
I20260809 03:25:11.689070 11155 tablet_bootstrap.cc:492] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39: No bootstrap required, opened a new log
I20260809 03:25:11.689170 11155 ts_tablet_manager.cc:1397] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:25:11.689603 11177 ts_tablet_manager.cc:1918] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:11.689684 11177 log.cc:1196] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/2c9ecd9a63a847f0b8ea81bd056fa091
I20260809 03:25:11.689909 11177 ts_tablet_manager.cc:1939] T 2c9ecd9a63a847f0b8ea81bd056fa091 P 1f91f5c79c024b60b91028e9b588cf39: Deleting consensus metadata
I20260809 03:25:11.690037 11155 raft_consensus.cc:348] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.690176 11155 raft_consensus.cc:374] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:11.690217 11155 raft_consensus.cc:729] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f91f5c79c024b60b91028e9b588cf39, State: Initialized, Role: FOLLOWER
I20260809 03:25:11.690373 11155 consensus_queue.cc:260] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.690726 11155 ts_tablet_manager.cc:1428] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:25:11.691762 11153 tablet_bootstrap.cc:492] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16: No bootstrap required, opened a new log
I20260809 03:25:11.692746 11154 tablet_bootstrap.cc:492] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5: No bootstrap required, opened a new log
I20260809 03:25:11.692755 11153 ts_tablet_manager.cc:1397] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260809 03:25:11.692834 11154 ts_tablet_manager.cc:1397] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260809 03:25:11.693388 11153 raft_consensus.cc:348] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.693506 11153 raft_consensus.cc:374] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:11.693538 11153 raft_consensus.cc:729] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df8ea0036fb44764b6a8ce36f6934e16, State: Initialized, Role: FOLLOWER
I20260809 03:25:11.693656 11153 consensus_queue.cc:260] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.693751 11154 raft_consensus.cc:348] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.693895 11154 raft_consensus.cc:374] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:11.693904 11153 ts_tablet_manager.cc:1428] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:11.693936 11154 raft_consensus.cc:729] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c4a490361441d5ad05d3544fad15a5, State: Initialized, Role: FOLLOWER
I20260809 03:25:11.694075 11154 consensus_queue.cc:260] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.694430 11154 ts_tablet_manager.cc:1428] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:11.695669 10599 catalog_manager.cc:4841] TS 1f91f5c79c024b60b91028e9b588cf39 (127.9.46.194:45169): tablet 2c9ecd9a63a847f0b8ea81bd056fa091 (table test-table [id=06eac51dd66c43ba9429228d18fd5602]) successfully deleted
I20260809 03:25:11.696151 11178 raft_consensus.cc:2229] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:11.696533 11178 raft_consensus.cc:2258] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:11.697110 11176 ts_tablet_manager.cc:1918] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:11.697553 11176 log.cc:1196] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/2c9ecd9a63a847f0b8ea81bd056fa091
I20260809 03:25:11.698093 11176 ts_tablet_manager.cc:1939] T 2c9ecd9a63a847f0b8ea81bd056fa091 P d1c4a490361441d5ad05d3544fad15a5: Deleting consensus metadata
I20260809 03:25:11.698428 11178 ts_tablet_manager.cc:1905] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:11.699342 10598 catalog_manager.cc:4841] TS d1c4a490361441d5ad05d3544fad15a5 (127.9.46.195:43647): tablet 2c9ecd9a63a847f0b8ea81bd056fa091 (table test-table [id=06eac51dd66c43ba9429228d18fd5602]) successfully deleted
I20260809 03:25:11.701293 11178 ts_tablet_manager.cc:1918] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:11.701467 11178 log.cc:1196] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/2c9ecd9a63a847f0b8ea81bd056fa091
I20260809 03:25:11.701879 11178 ts_tablet_manager.cc:1939] T 2c9ecd9a63a847f0b8ea81bd056fa091 P df8ea0036fb44764b6a8ce36f6934e16: Deleting consensus metadata
I20260809 03:25:11.702936 10601 catalog_manager.cc:4841] TS df8ea0036fb44764b6a8ce36f6934e16 (127.9.46.193:35185): tablet 2c9ecd9a63a847f0b8ea81bd056fa091 (table test-table [id=06eac51dd66c43ba9429228d18fd5602]) successfully deleted
I20260809 03:25:11.894830 11160 raft_consensus.cc:482] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:11.895001 11160 raft_consensus.cc:504] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.896312 11160 leader_election.cc:288] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df8ea0036fb44764b6a8ce36f6934e16 (127.9.46.193:35185), d1c4a490361441d5ad05d3544fad15a5 (127.9.46.195:43647)
I20260809 03:25:11.903023 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.897239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:55199 (local address 127.9.46.193:35185)
0809 03:25:11.897375 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.897379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.897580 (+ 201us) server_negotiation.cc:407] Connection header received
0809 03:25:11.897775 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.897778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.897822 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.897867 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:11.898387 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.899255 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.901259 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.901983 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.902087 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.902147 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.902772 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.902871 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:25:11.904095 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.896647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:35185 (local address 127.9.46.194:55199)
0809 03:25:11.897482 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.897496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.897657 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.898002 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.898005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.898021 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:11.898249 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.898258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.899437 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.899440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.901106 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.901117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.902326 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.902330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.902533 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.902567 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.903576 (+ 1009us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.903759 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":59,"thread_start_us":117,"threads_started":1}
I20260809 03:25:11.904369 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.897087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:57291 (local address 127.9.46.195:43647)
0809 03:25:11.897210 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.897215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.897780 (+ 565us) server_negotiation.cc:407] Connection header received
0809 03:25:11.898167 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.898171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.898246 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.898310 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:11.899024 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.899947 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.902169 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.903122 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.903370 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.903451 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.904075 (+ 624us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.904185 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:25:11.904831 10782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6786d40bcb8475b93f991b1e64b4b12" candidate_uuid: "1f91f5c79c024b60b91028e9b588cf39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df8ea0036fb44764b6a8ce36f6934e16" is_pre_election: true
I20260809 03:25:11.904994 10782 raft_consensus.cc:2454] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f91f5c79c024b60b91028e9b588cf39 in term 0.
I20260809 03:25:11.905169 11182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.896700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:43647 (local address 127.9.46.194:57291)
0809 03:25:11.897646 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.897665 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.897893 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.898488 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.898492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.898529 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:11.898831 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.898840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.900204 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.900209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.901966 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.901982 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.903578 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.903585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.903889 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.903927 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.904806 (+ 879us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.904917 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"mutex_wait_us":35,"thread_start_us":98,"threads_started":1}
I20260809 03:25:11.905565 10858 leader_election.cc:302] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f91f5c79c024b60b91028e9b588cf39, df8ea0036fb44764b6a8ce36f6934e16; no voters:
I20260809 03:25:11.905896 11160 raft_consensus.cc:2781] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:11.905956 11160 raft_consensus.cc:482] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:11.905998 11160 raft_consensus.cc:3037] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:11.905973 11077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6786d40bcb8475b93f991b1e64b4b12" candidate_uuid: "1f91f5c79c024b60b91028e9b588cf39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c4a490361441d5ad05d3544fad15a5" is_pre_election: true
I20260809 03:25:11.906211 11077 raft_consensus.cc:2454] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f91f5c79c024b60b91028e9b588cf39 in term 0.
I20260809 03:25:11.907624 11160 raft_consensus.cc:504] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.907984 11160 leader_election.cc:288] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [CANDIDATE]: Term 1 election: Requested vote from peers df8ea0036fb44764b6a8ce36f6934e16 (127.9.46.193:35185), d1c4a490361441d5ad05d3544fad15a5 (127.9.46.195:43647)
I20260809 03:25:11.908392 11077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6786d40bcb8475b93f991b1e64b4b12" candidate_uuid: "1f91f5c79c024b60b91028e9b588cf39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c4a490361441d5ad05d3544fad15a5"
I20260809 03:25:11.908463 10782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6786d40bcb8475b93f991b1e64b4b12" candidate_uuid: "1f91f5c79c024b60b91028e9b588cf39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df8ea0036fb44764b6a8ce36f6934e16"
I20260809 03:25:11.908567 11077 raft_consensus.cc:3037] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:11.908619 10782 raft_consensus.cc:3037] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:11.909826 10782 raft_consensus.cc:2454] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f91f5c79c024b60b91028e9b588cf39 in term 1.
I20260809 03:25:11.910152 10858 leader_election.cc:302] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f91f5c79c024b60b91028e9b588cf39, df8ea0036fb44764b6a8ce36f6934e16; no voters:
I20260809 03:25:11.910293 11077 raft_consensus.cc:2454] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f91f5c79c024b60b91028e9b588cf39 in term 1.
I20260809 03:25:11.910355 11160 raft_consensus.cc:2781] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:11.910604 11160 raft_consensus.cc:686] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 1 LEADER]: Becoming Leader. State: Replica: 1f91f5c79c024b60b91028e9b588cf39, State: Running, Role: LEADER
I20260809 03:25:11.910887 11160 consensus_queue.cc:237] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.913535 10614 catalog_manager.cc:5495] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f91f5c79c024b60b91028e9b588cf39 (127.9.46.194). New cstate: current_term: 1 leader_uuid: "1f91f5c79c024b60b91028e9b588cf39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } health_report { overall_health: UNKNOWN } } }
Aug 09 03:25:11 dist-test-slave-65mx krb5kdc[10567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245910, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:11.955862 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.946530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34099 (local address 127.0.0.1:34648)
0809 03:25:11.946945 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:11.946959 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:11.947061 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:11.947346 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:11.947349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:11.947805 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:11.947921 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.947927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.949086 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.949088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.949818 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:11.949827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.950564 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.950566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:11.950687 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.951300 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:11.951312 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:11.952839 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:11.954409 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:11.954417 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:11.954422 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:11.954665 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:11.954906 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:11.954908 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:11.954910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:11.954987 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:11.955339 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:11.955344 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:11.955445 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 03:25:11.955533 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 03:25:11.955674 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":181,"threads_started":1}
I20260809 03:25:11.955871 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:11.946583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34648 (local address 127.9.46.254:34099)
0809 03:25:11.946990 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:11.947009 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:11.947029 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:25:11.947151 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:11.947154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:11.947197 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:11.947280 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:11.948035 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.948962 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.949958 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:11.950434 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:11.950526 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:11.952973 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:11.952976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:11.952978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:11.952998 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:11.954307 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.954750 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.954753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.954754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.954826 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:11.955061 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:11.955063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:11.955065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:11.955250 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:11.955326 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:11.955588 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 03:25:11.955732 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":331,"thread_start_us":213,"threads_started":1}
I20260809 03:25:11.960091 10614 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:34648:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 03:25:11.960886 10614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:11.965852 10762 tablet_service.cc:1467] Processing CreateTablet for tablet 47a54a4f7e464a8ba5c20f2a4db9880b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57e1c238b0da492e9c7a762285f6861b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:25:11.966019 11054 tablet_service.cc:1467] Processing CreateTablet for tablet 47a54a4f7e464a8ba5c20f2a4db9880b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57e1c238b0da492e9c7a762285f6861b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:25:11.966019 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 47a54a4f7e464a8ba5c20f2a4db9880b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57e1c238b0da492e9c7a762285f6861b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:25:11.966356 10762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a54a4f7e464a8ba5c20f2a4db9880b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:11.966403 11054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a54a4f7e464a8ba5c20f2a4db9880b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:11.966403 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a54a4f7e464a8ba5c20f2a4db9880b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:11.969341 11154 tablet_bootstrap.cc:492] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: Bootstrap starting.
I20260809 03:25:11.969350 11153 tablet_bootstrap.cc:492] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: Bootstrap starting.
I20260809 03:25:11.969394 11155 tablet_bootstrap.cc:492] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: Bootstrap starting.
I20260809 03:25:11.970155 11153 tablet_bootstrap.cc:654] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:11.970492 11154 tablet_bootstrap.cc:654] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:11.970887 11155 tablet_bootstrap.cc:654] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:11.971840 11153 tablet_bootstrap.cc:492] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: No bootstrap required, opened a new log
I20260809 03:25:11.971900 11153 ts_tablet_manager.cc:1397] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 03:25:11.972738 11155 tablet_bootstrap.cc:492] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: No bootstrap required, opened a new log
I20260809 03:25:11.972663 11153 raft_consensus.cc:348] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.972788 11153 raft_consensus.cc:374] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:11.972829 11155 ts_tablet_manager.cc:1397] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:11.972863 11153 raft_consensus.cc:729] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df8ea0036fb44764b6a8ce36f6934e16, State: Initialized, Role: FOLLOWER
I20260809 03:25:11.973024 11153 consensus_queue.cc:260] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.973129 11154 tablet_bootstrap.cc:492] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: No bootstrap required, opened a new log
I20260809 03:25:11.973232 11154 ts_tablet_manager.cc:1397] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 03:25:11.973371 11153 ts_tablet_manager.cc:1428] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:11.973451 11155 raft_consensus.cc:348] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.973598 11155 raft_consensus.cc:374] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:11.973661 11155 raft_consensus.cc:729] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f91f5c79c024b60b91028e9b588cf39, State: Initialized, Role: FOLLOWER
I20260809 03:25:11.973654 11154 raft_consensus.cc:348] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.973774 11154 raft_consensus.cc:374] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:11.973824 11154 raft_consensus.cc:729] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c4a490361441d5ad05d3544fad15a5, State: Initialized, Role: FOLLOWER
I20260809 03:25:11.973892 11155 consensus_queue.cc:260] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.973953 11154 consensus_queue.cc:260] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:11.974231 11154 ts_tablet_manager.cc:1428] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:11.974257 11155 ts_tablet_manager.cc:1428] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:11.974220 11160 tablet_replica.cc:439] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } }
I20260809 03:25:11.974359 11160 tablet_replica.cc:442] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:11.974495 11159 tablet_replica.cc:439] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } }
I20260809 03:25:11.974628 11159 tablet_replica.cc:442] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:11.974815 11173 tablet_replica.cc:439] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } }
I20260809 03:25:11.974972 11173 tablet_replica.cc:442] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:12.198257 11159 raft_consensus.cc:482] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:12.198421 11159 raft_consensus.cc:504] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:12.199750 11159 leader_election.cc:288] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f91f5c79c024b60b91028e9b588cf39 (127.9.46.194:45169), d1c4a490361441d5ad05d3544fad15a5 (127.9.46.195:43647)
I20260809 03:25:12.204641 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.200113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:56399 (local address 127.9.46.194:45169)
0809 03:25:12.200462 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:12.200467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:12.200487 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:25:12.200706 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:12.200710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:12.200766 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:12.200864 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:12.201152 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.201914 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.203403 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.203934 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.204084 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.204145 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:12.204408 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 03:25:12.204492 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":75,"threads_started":1}
I20260809 03:25:12.204651 11196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.200072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:45169 (local address 127.9.46.193:56399)
0809 03:25:12.200388 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:12.200401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:12.200509 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:12.200875 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:12.200878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:12.200895 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:12.201037 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:12.201043 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.202035 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.202039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:12.203274 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:12.203286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.204110 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.204114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:12.204273 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.204293 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 03:25:12.204388 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 03:25:12.204466 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":40,"thread_start_us":75,"threads_started":1}
I20260809 03:25:12.204790 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.200193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:33357 (local address 127.9.46.195:43647)
0809 03:25:12.200324 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:12.200329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:12.200636 (+ 307us) server_negotiation.cc:407] Connection header received
0809 03:25:12.200760 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:12.200763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:12.200818 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:12.200906 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:12.201428 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.202093 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.203503 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.204044 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.204172 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.204226 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:12.204597 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 03:25:12.204678 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:25:12.204851 11197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.200088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:43647 (local address 127.9.46.193:33357)
0809 03:25:12.200520 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:12.200533 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:12.200672 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:12.201145 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:12.201148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:12.201167 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:12.201314 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:12.201320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.202235 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.202238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:12.203377 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:12.203387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.204173 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.204177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:12.204427 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.204448 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:25:12.204565 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 03:25:12.204642 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":16,"thread_start_us":85,"threads_started":1}
I20260809 03:25:12.205468 11077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a54a4f7e464a8ba5c20f2a4db9880b" candidate_uuid: "df8ea0036fb44764b6a8ce36f6934e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c4a490361441d5ad05d3544fad15a5" is_pre_election: true
I20260809 03:25:12.205478 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a54a4f7e464a8ba5c20f2a4db9880b" candidate_uuid: "df8ea0036fb44764b6a8ce36f6934e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f91f5c79c024b60b91028e9b588cf39" is_pre_election: true
I20260809 03:25:12.205646 11077 raft_consensus.cc:2454] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df8ea0036fb44764b6a8ce36f6934e16 in term 0.
I20260809 03:25:12.205657 10929 raft_consensus.cc:2454] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df8ea0036fb44764b6a8ce36f6934e16 in term 0.
I20260809 03:25:12.206063 10711 leader_election.cc:302] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1c4a490361441d5ad05d3544fad15a5, df8ea0036fb44764b6a8ce36f6934e16; no voters:
I20260809 03:25:12.206377 11159 raft_consensus.cc:2781] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:12.206447 11159 raft_consensus.cc:482] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:12.206494 11159 raft_consensus.cc:3037] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:12.208201 11159 raft_consensus.cc:504] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:12.208601 11159 leader_election.cc:288] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [CANDIDATE]: Term 1 election: Requested vote from peers 1f91f5c79c024b60b91028e9b588cf39 (127.9.46.194:45169), d1c4a490361441d5ad05d3544fad15a5 (127.9.46.195:43647)
I20260809 03:25:12.208983 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a54a4f7e464a8ba5c20f2a4db9880b" candidate_uuid: "df8ea0036fb44764b6a8ce36f6934e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f91f5c79c024b60b91028e9b588cf39"
I20260809 03:25:12.209023 11077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a54a4f7e464a8ba5c20f2a4db9880b" candidate_uuid: "df8ea0036fb44764b6a8ce36f6934e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c4a490361441d5ad05d3544fad15a5"
I20260809 03:25:12.209106 10929 raft_consensus.cc:3037] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:12.209167 11077 raft_consensus.cc:3037] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:12.210073 10929 raft_consensus.cc:2454] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df8ea0036fb44764b6a8ce36f6934e16 in term 1.
I20260809 03:25:12.210361 10712 leader_election.cc:302] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f91f5c79c024b60b91028e9b588cf39, df8ea0036fb44764b6a8ce36f6934e16; no voters:
I20260809 03:25:12.210383 11077 raft_consensus.cc:2454] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df8ea0036fb44764b6a8ce36f6934e16 in term 1.
I20260809 03:25:12.210531 11159 raft_consensus.cc:2781] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:12.210749 11159 raft_consensus.cc:686] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [term 1 LEADER]: Becoming Leader. State: Replica: df8ea0036fb44764b6a8ce36f6934e16, State: Running, Role: LEADER
I20260809 03:25:12.211007 11159 consensus_queue.cc:237] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } }
I20260809 03:25:12.212162 11199 tablet_replica.cc:439] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: TxnStatusTablet state changed. Reason: New leader df8ea0036fb44764b6a8ce36f6934e16. Latest consensus state: current_term: 1 leader_uuid: "df8ea0036fb44764b6a8ce36f6934e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } }
I20260809 03:25:12.212271 11199 tablet_replica.cc:442] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:25:12.212514 11201 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 03:25:12.212704 11201 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 03:25:12.213678 10614 catalog_manager.cc:5495] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 reported cstate change: term changed from 0 to 1, leader changed from <none> to df8ea0036fb44764b6a8ce36f6934e16 (127.9.46.193). New cstate: current_term: 1 leader_uuid: "df8ea0036fb44764b6a8ce36f6934e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } health_report { overall_health: UNKNOWN } } }
I20260809 03:25:12.268970 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.264445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:35185 (local address 127.0.0.1:56354)
0809 03:25:12.264734 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:12.264750 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:12.264859 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:12.265231 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:12.265234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:12.265244 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:12.265345 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:12.265350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.266473 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.266476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:12.267105 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:12.267113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.267690 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.267692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:12.267763 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.267818 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:12.268666 (+ 848us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:12.268683 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 03:25:12.268754 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 03:25:12.268833 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260809 03:25:12.269065 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.264626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.9.46.193:35185)
0809 03:25:12.264756 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:12.264761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:12.264802 (+ 41us) server_negotiation.cc:407] Connection header received
0809 03:25:12.265007 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:12.265010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:12.265078 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:12.265187 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:12.265464 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.266350 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.267226 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.267578 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.267680 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.267961 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:12.268512 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:12.268653 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:12.268768 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 03:25:12.268833 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 03:25:12.272495 10929 raft_consensus.cc:1261] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39 [term 1 FOLLOWER]: Refusing update from remote peer df8ea0036fb44764b6a8ce36f6934e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:12.272851 11077 raft_consensus.cc:1261] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5 [term 1 FOLLOWER]: Refusing update from remote peer df8ea0036fb44764b6a8ce36f6934e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:12.273161 11199 consensus_queue.cc:1035] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:12.273368 11159 consensus_queue.cc:1035] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:12.274863 11160 tablet_replica.cc:439] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: TxnStatusTablet state changed. Reason: New leader df8ea0036fb44764b6a8ce36f6934e16. Latest consensus state: current_term: 1 leader_uuid: "df8ea0036fb44764b6a8ce36f6934e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } }
I20260809 03:25:12.274981 11160 tablet_replica.cc:442] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:12.275470 11173 tablet_replica.cc:439] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: TxnStatusTablet state changed. Reason: New leader df8ea0036fb44764b6a8ce36f6934e16. Latest consensus state: current_term: 1 leader_uuid: "df8ea0036fb44764b6a8ce36f6934e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } }
I20260809 03:25:12.275584 11173 tablet_replica.cc:442] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:12.278164 11159 tablet_replica.cc:439] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df8ea0036fb44764b6a8ce36f6934e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } }
I20260809 03:25:12.278282 11159 tablet_replica.cc:442] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:25:12.278240 11199 tablet_replica.cc:439] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "df8ea0036fb44764b6a8ce36f6934e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } }
I20260809 03:25:12.278404 11199 tablet_replica.cc:442] T 47a54a4f7e464a8ba5c20f2a4db9880b P df8ea0036fb44764b6a8ce36f6934e16: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:25:12.279078 11160 tablet_replica.cc:439] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df8ea0036fb44764b6a8ce36f6934e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } }
I20260809 03:25:12.279193 11160 tablet_replica.cc:442] T 47a54a4f7e464a8ba5c20f2a4db9880b P 1f91f5c79c024b60b91028e9b588cf39: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:12.279651 11173 tablet_replica.cc:439] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "df8ea0036fb44764b6a8ce36f6934e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 } } peers { permanent_uuid: "1f91f5c79c024b60b91028e9b588cf39" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 45169 } } peers { permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 } } }
I20260809 03:25:12.279747 11173 tablet_replica.cc:442] T 47a54a4f7e464a8ba5c20f2a4db9880b P d1c4a490361441d5ad05d3544fad15a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:12.280648 11209 mvcc.cc:205] Tried to move back new op lower bound from 7316463256662986752 to 7316463256418680832. Current Snapshot: MvccSnapshot[applied={T|T < 7316463256662986752}]
I20260809 03:25:12.290669 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.286834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.9.46.194:45169)
0809 03:25:12.286974 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:12.287004 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:12.287161 (+ 157us) server_negotiation.cc:407] Connection header received
0809 03:25:12.287354 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:12.287356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:12.287413 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:12.287532 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:12.287796 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.288302 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.289222 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.289439 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.289554 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.289833 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:12.290272 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:12.290403 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:12.290447 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 03:25:12.290511 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:25:12.303076 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.298367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:44579 (local address 127.9.46.193:35185)
0809 03:25:12.298599 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:12.298604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:12.298780 (+ 176us) server_negotiation.cc:407] Connection header received
0809 03:25:12.298859 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:12.298862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:12.298916 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:12.299010 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:12.299452 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.300476 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.301528 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.301865 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.301973 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.302524 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:12.302608 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:12.302759 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:12.302875 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 03:25:12.302928 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260809 03:25:12.303377 11219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.298184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:35185 (local address 127.9.46.194:44579)
0809 03:25:12.298672 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:12.298686 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:12.298780 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:12.299069 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:12.299072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:12.299083 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:12.299323 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:12.299334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.300662 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.300665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:12.301375 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:12.301385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.301980 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.301982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:12.302303 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.302368 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:12.302782 (+ 414us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:12.302788 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:25:12.302837 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 03:25:12.302905 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":149,"threads_started":1}
I20260809 03:25:12.309820 11077 raft_consensus.cc:1261] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5 [term 1 FOLLOWER]: Refusing update from remote peer 1f91f5c79c024b60b91028e9b588cf39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:12.309820 10782 raft_consensus.cc:1261] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16 [term 1 FOLLOWER]: Refusing update from remote peer 1f91f5c79c024b60b91028e9b588cf39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:12.310298 11160 consensus_queue.cc:1035] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c4a490361441d5ad05d3544fad15a5" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:12.310525 11184 consensus_queue.cc:1035] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df8ea0036fb44764b6a8ce36f6934e16" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:12.342377 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.336733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:40865 (local address 127.9.46.194:45169)
0809 03:25:12.336842 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:12.336847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:12.337364 (+ 517us) server_negotiation.cc:407] Connection header received
0809 03:25:12.337542 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:12.337545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:12.337591 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:12.337650 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:12.338176 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.338994 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.340779 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.341111 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.341199 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.341895 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:12.341985 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:12.342030 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:12.342185 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 03:25:12.342238 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 03:25:12.343210 11223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.336767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:45169 (local address 127.9.46.193:40865)
0809 03:25:12.337279 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:12.337295 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:12.337435 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:12.337781 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:12.337783 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:12.337791 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:12.338046 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:12.338054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.339230 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.339233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:12.340645 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:12.340657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.341550 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.341554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:12.341675 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.341729 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:12.342093 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:12.342099 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:25:12.342774 (+ 675us) client_negotiation.cc:240] Negotiation successful
0809 03:25:12.342866 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":245,"threads_started":1}
I20260809 03:25:12.378427 10614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:12.378626 10614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:12.381336 10614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfa804c3390048a78947be90c1bb54f0: Sending DeleteTablet for 3 replicas of tablet b6786d40bcb8475b93f991b1e64b4b12
I20260809 03:25:12.382037 10762 tablet_service.cc:1514] Processing DeleteTablet for tablet b6786d40bcb8475b93f991b1e64b4b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:12 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:56338
I20260809 03:25:12.382254 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet b6786d40bcb8475b93f991b1e64b4b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:12 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:56244
I20260809 03:25:12.382709 11229 tablet_replica.cc:330] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16: stopping tablet replica
I20260809 03:25:12.382866 11229 raft_consensus.cc:2229] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:12.382990 11229 raft_consensus.cc:2258] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:12.383039 11054 tablet_service.cc:1514] Processing DeleteTablet for tablet b6786d40bcb8475b93f991b1e64b4b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:12 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:57784
I20260809 03:25:12.383847 11229 ts_tablet_manager.cc:1905] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:12.384720 11230 tablet_replica.cc:330] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39: stopping tablet replica
I20260809 03:25:12.384913 11230 raft_consensus.cc:2229] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:12.385231 11230 raft_consensus.cc:2258] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:12.385507 11231 tablet_replica.cc:330] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5: stopping tablet replica
I20260809 03:25:12.385663 11231 raft_consensus.cc:2229] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:12.385800 11231 raft_consensus.cc:2258] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:12.386592 11231 ts_tablet_manager.cc:1905] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:12.386945 11230 ts_tablet_manager.cc:1905] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:12.390125 11231 ts_tablet_manager.cc:1918] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:12.390268 11231 log.cc:1196] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/b6786d40bcb8475b93f991b1e64b4b12
I20260809 03:25:12.390628 11230 ts_tablet_manager.cc:1918] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:12.390714 11230 log.cc:1196] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/b6786d40bcb8475b93f991b1e64b4b12
I20260809 03:25:12.390874 11231 ts_tablet_manager.cc:1939] T b6786d40bcb8475b93f991b1e64b4b12 P d1c4a490361441d5ad05d3544fad15a5: Deleting consensus metadata
I20260809 03:25:12.391814 10598 catalog_manager.cc:4841] TS d1c4a490361441d5ad05d3544fad15a5 (127.9.46.195:43647): tablet b6786d40bcb8475b93f991b1e64b4b12 (table test-table [id=9c4946f46ac540c0bf49b5ad229da25f]) successfully deleted
I20260809 03:25:12.393780 11229 ts_tablet_manager.cc:1918] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:12.393872 11229 log.cc:1196] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/b6786d40bcb8475b93f991b1e64b4b12
I20260809 03:25:12.394260 11229 ts_tablet_manager.cc:1939] T b6786d40bcb8475b93f991b1e64b4b12 P df8ea0036fb44764b6a8ce36f6934e16: Deleting consensus metadata
I20260809 03:25:12.395083 11230 ts_tablet_manager.cc:1939] T b6786d40bcb8475b93f991b1e64b4b12 P 1f91f5c79c024b60b91028e9b588cf39: Deleting consensus metadata
I20260809 03:25:12.395247 10601 catalog_manager.cc:4841] TS df8ea0036fb44764b6a8ce36f6934e16 (127.9.46.193:35185): tablet b6786d40bcb8475b93f991b1e64b4b12 (table test-table [id=9c4946f46ac540c0bf49b5ad229da25f]) successfully deleted
I20260809 03:25:12.396129 10599 catalog_manager.cc:4841] TS 1f91f5c79c024b60b91028e9b588cf39 (127.9.46.194:45169): tablet b6786d40bcb8475b93f991b1e64b4b12 (table test-table [id=9c4946f46ac540c0bf49b5ad229da25f]) successfully deleted
Aug 09 03:25:12 dist-test-slave-65mx krb5kdc[10567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245911, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:25:12.413302 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.386318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.9.46.193:35185)
0809 03:25:12.386444 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:12.386449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:12.396313 (+ 9864us) server_negotiation.cc:407] Connection header received
0809 03:25:12.396619 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:12.396622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:12.396678 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:12.396778 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:12.397711 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.398556 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.400155 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.400572 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.400681 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.408904 (+ 8223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:12.408918 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:12.408935 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:12.408972 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:12.411182 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:12.411748 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:12.411755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:12.411757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:12.411852 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:12.412139 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:12.412142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:12.412144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:12.412498 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:12.412861 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:12.412957 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 03:25:12.413099 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 03:25:12.413961 10825 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56370
I20260809 03:25:12.426056 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:12.416070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.9.46.254:34099)
0809 03:25:12.416637 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:12.416642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:12.416720 (+ 78us) server_negotiation.cc:407] Connection header received
0809 03:25:12.416942 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:12.416955 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:12.417031 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:12.417121 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:12.418057 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.418928 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.420134 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:12.420453 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:12.420601 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:12.422239 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:12.422243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:12.422246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:12.422276 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:12.423871 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:12.424414 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:12.424416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:12.424417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:12.424498 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:12.424902 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:12.424904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:12.424905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:12.425201 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:12.425313 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:12.425670 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 03:25:12.425914 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412}
W20260809 03:25:12.426507 10614 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34662
I20260809 03:25:12.427968 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 10694
I20260809 03:25:12.440503 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 10840
I20260809 03:25:12.448295 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 10987
I20260809 03:25:12.457577 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 10582
2026-08-09T03:25:12Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3267 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:12 dist-test-slave-65mx krb5kdc[11241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:12 dist-test-slave-65mx krb5kdc[11241](info): set up 2 sockets
Aug 09 03:25:12 dist-test-slave-65mx krb5kdc[11241](info): commencing operation
krb5kdc: starting...
W20260809 03:25:13.549741 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:13 dist-test-slave-65mx krb5kdc[11241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:13.684991 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43101
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:43101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:13.797920 11256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:13.798219 11256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:13.798262 11256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:13.802503 11256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:13.802587 11256 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 03:25:13.802606 11256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:13.802625 11256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:13.802670 11256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:13.802748 11256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:13.808379 11256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:43101
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:13.809551 11256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:13.810606 11256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:13.819548 11262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:13.819576 11265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:13.819614 11256 server_base.cc:1034] running on GCE node
W20260809 03:25:13.819551 11263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:13.820334 11256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:13.821594 11256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:13.822782 11256 hybrid_clock.cc:648] HybridClock initialized: now 1786245913822751 us; error 43 us; skew 500 ppm
Aug 09 03:25:13 dist-test-slave-65mx krb5kdc[11241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245913, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:13.829113 11256 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:13.830078 11256 webserver.cc:460] Webserver started at http://127.9.46.254:44441/ using document root <none> and password file <none>
I20260809 03:25:13.830648 11256 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:13.830711 11256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:13.830879 11256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:13.832958 11256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "7136bfe290754f8f99488e8fd223f73d"
format_stamp: "Formatted at 2026-08-09 03:25:13 on dist-test-slave-65mx"
server_key: "892d6807c8beab763fdf0750248b09f7"
server_key_iv: "044b0a668c8fbfca22561d42dcc00cbc"
server_key_version: "encryptionkey@0"
I20260809 03:25:13.833479 11256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "7136bfe290754f8f99488e8fd223f73d"
format_stamp: "Formatted at 2026-08-09 03:25:13 on dist-test-slave-65mx"
server_key: "892d6807c8beab763fdf0750248b09f7"
server_key_iv: "044b0a668c8fbfca22561d42dcc00cbc"
server_key_version: "encryptionkey@0"
I20260809 03:25:13.837452 11256 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:25:13.839825 11272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:13.841205 11256 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:13.841352 11256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "7136bfe290754f8f99488e8fd223f73d"
format_stamp: "Formatted at 2026-08-09 03:25:13 on dist-test-slave-65mx"
server_key: "892d6807c8beab763fdf0750248b09f7"
server_key_iv: "044b0a668c8fbfca22561d42dcc00cbc"
server_key_version: "encryptionkey@0"
I20260809 03:25:13.841470 11256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:13.856302 11256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:13.860019 11256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:13.860186 11256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:13.868235 11256 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:43101
I20260809 03:25:13.868287 11336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:43101 every 8 connection(s)
I20260809 03:25:13.869318 11256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:13.872284 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 11256
I20260809 03:25:13.872316 11337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:13.872421 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:13.872761 9403 external_mini_cluster.cc:1428] Setting key a307422de294815c15f52d7a0ea123dd
I20260809 03:25:13.879020 11337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d: Bootstrap starting.
I20260809 03:25:13.881354 11337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d: Neither blocks nor log segments found. Creating new log.
Aug 09 03:25:13 dist-test-slave-65mx krb5kdc[11241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:13.882157 11337 log.cc:824] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:13.884052 11337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d: No bootstrap required, opened a new log
I20260809 03:25:13.887014 11337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7136bfe290754f8f99488e8fd223f73d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43101 } }
I20260809 03:25:13.887267 11337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:13.887333 11337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7136bfe290754f8f99488e8fd223f73d, State: Initialized, Role: FOLLOWER
I20260809 03:25:13.887804 11337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7136bfe290754f8f99488e8fd223f73d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43101 } }
I20260809 03:25:13.887965 11337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:13.888047 11337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:13.888186 11337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:13.888787 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:13.874316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.9.46.254:43101)
0809 03:25:13.874808 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:13.874823 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:13.874860 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:25:13.875715 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:13.875735 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:13.876069 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:13.876414 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:13.877310 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:13.878289 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:13.879156 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:13.879636 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:13.879811 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:13.882518 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:13.882523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:13.882536 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:13.882565 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:13.885308 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:13.885767 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:13.885779 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:13.885783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:13.885945 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:13.886245 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:13.886248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:13.886250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:13.886649 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:13.886878 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:13.887520 (+ 642us) server_negotiation.cc:299] Negotiation successful
0809 03:25:13.887770 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":154,"threads_started":1}
I20260809 03:25:13.889364 11337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7136bfe290754f8f99488e8fd223f73d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43101 } }
I20260809 03:25:13.889717 11337 leader_election.cc:302] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7136bfe290754f8f99488e8fd223f73d; no voters:
I20260809 03:25:13.890038 11337 leader_election.cc:288] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:13.890179 11342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:13.890401 11342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [term 1 LEADER]: Becoming Leader. State: Replica: 7136bfe290754f8f99488e8fd223f73d, State: Running, Role: LEADER
I20260809 03:25:13.890697 11342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7136bfe290754f8f99488e8fd223f73d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43101 } }
I20260809 03:25:13.891391 11337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:13.891875 11344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7136bfe290754f8f99488e8fd223f73d. Latest consensus state: current_term: 1 leader_uuid: "7136bfe290754f8f99488e8fd223f73d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7136bfe290754f8f99488e8fd223f73d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43101 } } }
I20260809 03:25:13.892002 11344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:13.892411 11343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7136bfe290754f8f99488e8fd223f73d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7136bfe290754f8f99488e8fd223f73d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43101 } } }
I20260809 03:25:13.892513 11343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d [sys.catalog]: This master's current role is: LEADER
W20260809 03:25:13.896216 11360 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:25:13.896314 11360 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:25:13.896400 11347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:13.897163 11347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:13.902513 11347 catalog_manager.cc:1347] Generated new cluster ID: e385a7d4a1b44ea598969baa1d9cc73e
I20260809 03:25:13.902591 11347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:13.919939 11347 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:13.921211 11347 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:13.932276 11347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d: Generated new TSK 0
I20260809 03:25:13.933122 11347 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:25:13.968084 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43101
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:14.081679 11366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:14.081981 11366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:14.082100 11366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:14.086290 11366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:14.086369 11366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:14.086443 11366 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 03:25:14.086480 11366 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:25:14.086527 11366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:14.091727 11366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.46.254:43101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:14.092940 11366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:14.093978 11366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:14.103703 11371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:14.103781 11374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:14.103730 11372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:14.103900 11366 server_base.cc:1034] running on GCE node
I20260809 03:25:14.104512 11366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:14.105182 11366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:14.106357 11366 hybrid_clock.cc:648] HybridClock initialized: now 1786245914106327 us; error 41 us; skew 500 ppm
Aug 09 03:25:14 dist-test-slave-65mx krb5kdc[11241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245914, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:14.109313 11366 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:25:14.110213 11366 webserver.cc:460] Webserver started at http://127.9.46.193:44303/ using document root <none> and password file <none>
I20260809 03:25:14.110771 11366 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:14.110863 11366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:14.111132 11366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:14.113235 11366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "bc863088131c41cd83b6ef0f88a6faae"
format_stamp: "Formatted at 2026-08-09 03:25:14 on dist-test-slave-65mx"
server_key: "304c8358f9e04ef56709180a03d8be2e"
server_key_iv: "2a30f99215b523b87d358bb28097d1b1"
server_key_version: "encryptionkey@0"
I20260809 03:25:14.113797 11366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "bc863088131c41cd83b6ef0f88a6faae"
format_stamp: "Formatted at 2026-08-09 03:25:14 on dist-test-slave-65mx"
server_key: "304c8358f9e04ef56709180a03d8be2e"
server_key_iv: "2a30f99215b523b87d358bb28097d1b1"
server_key_version: "encryptionkey@0"
I20260809 03:25:14.117594 11366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:25:14.119884 11381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:14.121110 11366 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:14.121248 11366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "bc863088131c41cd83b6ef0f88a6faae"
format_stamp: "Formatted at 2026-08-09 03:25:14 on dist-test-slave-65mx"
server_key: "304c8358f9e04ef56709180a03d8be2e"
server_key_iv: "2a30f99215b523b87d358bb28097d1b1"
server_key_version: "encryptionkey@0"
I20260809 03:25:14.121357 11366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:14.136022 11366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:14.138827 11366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:14.139029 11366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:14.140487 11366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:14.140576 11366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:14.140653 11366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:14.140707 11366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:25:14 dist-test-slave-65mx krb5kdc[11241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245914, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:14.151682 11366 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:45387
I20260809 03:25:14.151810 11502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:45387 every 8 connection(s)
I20260809 03:25:14.152702 11366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:14.155584 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 11366
I20260809 03:25:14.155723 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:14.155987 9403 external_mini_cluster.cc:1428] Setting key 1a66a972d3ca64df4d23322029f29404
I20260809 03:25:14.156002 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.141300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:43415 (local address 127.9.46.254:43101)
0809 03:25:14.141429 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.141432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.143143 (+ 1711us) server_negotiation.cc:407] Connection header received
0809 03:25:14.144474 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.144478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.144569 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.144683 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:14.146310 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.147226 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.148901 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.149154 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.149287 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.152346 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:14.152352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:14.152355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:14.152384 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:14.154002 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.154457 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.154460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.154461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.154531 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.154919 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.154921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.154922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.155092 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:14.155167 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.155666 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.155814 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:25:14.156905 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.142360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43101 (local address 127.9.46.193:43415)
0809 03:25:14.142949 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.143004 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.144132 (+ 1128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.144852 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.144861 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.145451 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:14.146157 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.146178 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.147363 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.147366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.148779 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.148789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.149326 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.149330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.149536 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.150204 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:14.150230 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:14.152161 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:14.154104 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.154108 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.154118 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.154370 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.154673 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.154676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.154678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.154829 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.155190 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:14.155195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:14.155408 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.155620 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.155922 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":151,"threads_started":1}
I20260809 03:25:14.158414 11503 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43101
I20260809 03:25:14.158579 11503 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:14.159165 11503 heartbeater.cc:511] Master 127.9.46.254:43101 requested a full tablet report, sending...
I20260809 03:25:14.160794 11285 ts_manager.cc:194] Registered new tserver with Master: bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387)
I20260809 03:25:14.162308 11285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:43415
I20260809 03:25:14.168074 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.160899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:35305 (local address 127.9.46.254:43101)
0809 03:25:14.161012 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.161018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.161135 (+ 117us) server_negotiation.cc:407] Connection header received
0809 03:25:14.161256 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.161259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.161300 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.161381 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:14.161766 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.162487 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.163455 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.163699 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.163810 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.164982 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:14.164986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:14.164989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:14.165013 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:14.166452 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.166887 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.166891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.166893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.166970 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.167269 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.167272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.167274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.167440 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:14.167548 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.167805 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.167913 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 03:25:14.168118 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.160574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43101 (local address 127.9.46.193:35305)
0809 03:25:14.160900 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.160917 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.161160 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.161389 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.161391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.161553 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:14.161656 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.161661 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.162606 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.162609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.163334 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.163343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.163815 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.163818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.163912 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.164441 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:14.164452 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:14.164843 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:14.166585 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.166588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.166590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.166785 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.167082 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.167084 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.167086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.167171 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.167548 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:14.167552 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:14.167647 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.167753 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.167907 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:25:14.187178 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43101
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:14.299224 11513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:14.299486 11513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:14.299566 11513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:14.303668 11513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:14.303721 11513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:14.303761 11513 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 03:25:14.303815 11513 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:25:14.303874 11513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:14.309209 11513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.46.254:43101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:14.310227 11513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:14.311236 11513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:14.320976 11521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:14.321064 11513 server_base.cc:1034] running on GCE node
W20260809 03:25:14.320996 11519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:14.320976 11518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:14.321669 11513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:14.322283 11513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:14.323469 11513 hybrid_clock.cc:648] HybridClock initialized: now 1786245914323452 us; error 30 us; skew 500 ppm
Aug 09 03:25:14 dist-test-slave-65mx krb5kdc[11241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245914, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:14.326361 11513 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:25:14.327154 11513 webserver.cc:460] Webserver started at http://127.9.46.194:46673/ using document root <none> and password file <none>
I20260809 03:25:14.327677 11513 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:14.327734 11513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:14.327888 11513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:14.329864 11513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8"
format_stamp: "Formatted at 2026-08-09 03:25:14 on dist-test-slave-65mx"
server_key: "9c7bc335ca2189c6b7bd476730bd3bbd"
server_key_iv: "62c59d4bd80efb805f075d0f0b737505"
server_key_version: "encryptionkey@0"
I20260809 03:25:14.330358 11513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8"
format_stamp: "Formatted at 2026-08-09 03:25:14 on dist-test-slave-65mx"
server_key: "9c7bc335ca2189c6b7bd476730bd3bbd"
server_key_iv: "62c59d4bd80efb805f075d0f0b737505"
server_key_version: "encryptionkey@0"
I20260809 03:25:14.333981 11513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:25:14.336251 11528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:14.337473 11513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:14.337594 11513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8"
format_stamp: "Formatted at 2026-08-09 03:25:14 on dist-test-slave-65mx"
server_key: "9c7bc335ca2189c6b7bd476730bd3bbd"
server_key_iv: "62c59d4bd80efb805f075d0f0b737505"
server_key_version: "encryptionkey@0"
I20260809 03:25:14.337714 11513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:14.350636 11513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:14.353628 11513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:14.353868 11513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:14.355276 11513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:14.355353 11513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:14.355422 11513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:14.355471 11513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:25:14 dist-test-slave-65mx krb5kdc[11241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245914, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:14.367583 11513 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:36035
I20260809 03:25:14.367718 11648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:36035 every 8 connection(s)
I20260809 03:25:14.368706 11513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:14.370267 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.356261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:41937 (local address 127.9.46.254:43101)
0809 03:25:14.356396 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.356400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.357470 (+ 1070us) server_negotiation.cc:407] Connection header received
0809 03:25:14.358566 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.358570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.358634 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.358721 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:14.360135 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.360740 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.362105 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.362334 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.362449 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.366109 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:14.366113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:14.366116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:14.366149 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:14.367955 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.368622 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.368626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.368628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.368724 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.369142 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.369145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.369147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.369346 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:14.369428 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.370032 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.370144 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:25:14.371035 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.356800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43101 (local address 127.9.46.194:41937)
0809 03:25:14.357309 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.357351 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.358335 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.358886 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.358897 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.359439 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:14.359973 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.359987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.360865 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.360869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.361966 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.361978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.362511 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.362516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.362684 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.363353 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:14.363370 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:14.365906 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:14.368070 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.368076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.368088 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.368438 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.368840 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.368844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.368846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.368991 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.369456 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:14.369462 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:14.369757 (+ 295us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.369965 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.370228 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":162,"threads_started":1}
I20260809 03:25:14.372277 11649 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43101
I20260809 03:25:14.372437 11649 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:14.373021 11649 heartbeater.cc:511] Master 127.9.46.254:43101 requested a full tablet report, sending...
I20260809 03:25:14.374074 11287 ts_manager.cc:194] Registered new tserver with Master: 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194:36035)
I20260809 03:25:14.374230 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 11513
I20260809 03:25:14.374341 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:14.374631 9403 external_mini_cluster.cc:1428] Setting key b651e91fe00ba3ec9d976d4d1a971197
I20260809 03:25:14.374855 11287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:41937
I20260809 03:25:14.382009 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.374424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:59811 (local address 127.9.46.254:43101)
0809 03:25:14.374539 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.374543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.375277 (+ 734us) server_negotiation.cc:407] Connection header received
0809 03:25:14.375393 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.375396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.375445 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.375526 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:14.376032 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.376529 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.377404 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.377627 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.377737 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.378826 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:14.378829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:14.378832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:14.378856 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:14.380271 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.380777 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.380780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.380782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.380865 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.381145 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.381149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.381151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.381362 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:14.381475 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.381747 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.381857 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:25:14.382017 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.374370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43101 (local address 127.9.46.194:59811)
0809 03:25:14.375179 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.375195 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.375298 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.375640 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.375642 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.375818 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:14.375915 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.375920 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.376709 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.376712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.377282 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.377290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.377740 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.377744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.377846 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.378356 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:14.378368 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:14.378688 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:14.380390 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.380393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.380395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.380670 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.380958 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.380962 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.380964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.381053 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.381464 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:14.381467 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:14.381572 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.381677 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.381818 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:25:14.401867 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43101
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:14.512811 11658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:14.513070 11658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:14.513125 11658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:14.517167 11658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:14.517221 11658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:14.517297 11658 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 03:25:14.517329 11658 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:25:14.517396 11658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:14.522686 11658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.46.254:43101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:14.523702 11658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:14.524745 11658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:14.534353 11665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:14.534353 11668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:14.534391 11664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:14.534701 11658 server_base.cc:1034] running on GCE node
I20260809 03:25:14.535249 11658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:14.535862 11658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:14.537065 11658 hybrid_clock.cc:648] HybridClock initialized: now 1786245914537021 us; error 47 us; skew 500 ppm
Aug 09 03:25:14 dist-test-slave-65mx krb5kdc[11241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245914, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:14.539776 11658 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:25:14.540678 11658 webserver.cc:460] Webserver started at http://127.9.46.195:39751/ using document root <none> and password file <none>
I20260809 03:25:14.541199 11658 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:14.541258 11658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:14.541420 11658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:14.543417 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "f9ff3d127ff64be29e732807d0abf08c"
format_stamp: "Formatted at 2026-08-09 03:25:14 on dist-test-slave-65mx"
server_key: "88cae9a79bd6bf7ec1edb13dddbdabfd"
server_key_iv: "6945fe73d5d428411a9da18b599ae9bb"
server_key_version: "encryptionkey@0"
I20260809 03:25:14.543922 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "f9ff3d127ff64be29e732807d0abf08c"
format_stamp: "Formatted at 2026-08-09 03:25:14 on dist-test-slave-65mx"
server_key: "88cae9a79bd6bf7ec1edb13dddbdabfd"
server_key_iv: "6945fe73d5d428411a9da18b599ae9bb"
server_key_version: "encryptionkey@0"
I20260809 03:25:14.547516 11658 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 03:25:14.549857 11675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:14.551016 11658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:25:14.551146 11658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "f9ff3d127ff64be29e732807d0abf08c"
format_stamp: "Formatted at 2026-08-09 03:25:14 on dist-test-slave-65mx"
server_key: "88cae9a79bd6bf7ec1edb13dddbdabfd"
server_key_iv: "6945fe73d5d428411a9da18b599ae9bb"
server_key_version: "encryptionkey@0"
I20260809 03:25:14.551232 11658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:14.570309 11658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:14.573427 11658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:14.573585 11658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:14.574932 11658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:14.575012 11658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:14.575073 11658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:14.575104 11658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:25:14 dist-test-slave-65mx krb5kdc[11241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245914, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:14.585631 11658 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:45377
I20260809 03:25:14.585716 11795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:45377 every 8 connection(s)
I20260809 03:25:14.586774 11658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:14.588335 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 11658
I20260809 03:25:14.588476 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:14.588794 9403 external_mini_cluster.cc:1428] Setting key a2e0c38db1fc9554ebc79b17f79781d7
I20260809 03:25:14.589254 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.575779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:38401 (local address 127.9.46.254:43101)
0809 03:25:14.575905 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.575909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.576975 (+ 1066us) server_negotiation.cc:407] Connection header received
0809 03:25:14.578172 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.578177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.578247 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.578349 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:14.579875 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.580825 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.582165 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.582520 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.582637 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.585283 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:14.585287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:14.585290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:14.585325 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:14.587003 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.587619 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.587625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.587627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.587714 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.588077 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.588080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.588082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.588266 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:14.588377 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.588951 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.589083 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:25:14.590106 11685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.576296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43101 (local address 127.9.46.195:38401)
0809 03:25:14.576818 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.576860 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.577786 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.578543 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.578552 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.578997 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:14.579718 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.579736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.580963 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.580969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.582013 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.582024 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.582667 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.582670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.582806 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.583360 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:14.583375 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:14.585082 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:14.587154 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.587161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.587179 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.587513 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.587818 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.587823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.587826 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.587971 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.588375 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:14.588381 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:14.588711 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.588888 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.589187 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":163,"threads_started":1}
I20260809 03:25:14.591488 11797 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43101
I20260809 03:25:14.591651 11797 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:14.592232 11797 heartbeater.cc:511] Master 127.9.46.254:43101 requested a full tablet report, sending...
I20260809 03:25:14.593428 11287 ts_manager.cc:194] Registered new tserver with Master: f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377)
I20260809 03:25:14.594208 11287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:38401
I20260809 03:25:14.600524 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.593735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:40987 (local address 127.9.46.254:43101)
0809 03:25:14.593826 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.593829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.594064 (+ 235us) server_negotiation.cc:407] Connection header received
0809 03:25:14.594230 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.594234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.594277 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.594351 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:14.594801 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.595302 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.596142 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.596362 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.596470 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.597601 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:14.597604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:14.597607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:14.597631 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:14.598874 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.599323 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.599326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.599328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.599415 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.599705 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.599710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.599712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.599887 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:14.599996 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.600262 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.600372 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 03:25:14.600533 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.593681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43101 (local address 127.9.46.195:40987)
0809 03:25:14.593974 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.593986 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.594128 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.594365 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.594368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.594582 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:14.594690 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.594696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.595429 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.595432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.596010 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.596017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.596473 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.596477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.596598 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.597151 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:14.597161 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:14.597457 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:14.598989 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.598994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.598996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.599213 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.599517 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.599520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.599522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.599610 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.599986 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:14.599989 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:14.600096 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.600205 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.600347 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":114,"threads_started":1}
I20260809 03:25:14.602694 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:25:14 dist-test-slave-65mx krb5kdc[11241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:25:14 dist-test-slave-65mx krb5kdc[11241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:14.630550 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.622092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50258 (local address 127.9.46.254:43101)
0809 03:25:14.622255 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.622260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.622276 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:25:14.622323 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.622325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.622363 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.622452 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:14.623192 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.623731 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.624734 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.624962 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.625075 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.627531 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:14.627535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:14.627539 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:14.627583 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:14.628822 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.629286 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.629295 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.629296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.629375 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.629663 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.629667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.629669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.629861 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:14.630004 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.630250 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.630376 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:25:14.633530 11287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:14.636468 11287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:14.651199 11820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.645441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55026 (local address 127.9.46.193:45387)
0809 03:25:14.645999 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.646016 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.646265 (+ 249us) server_negotiation.cc:407] Connection header received
0809 03:25:14.646489 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.646494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.646701 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.646826 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:14.647230 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.648134 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.649500 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.650333 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.650455 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.650624 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.650936 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.651033 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":105,"threads_started":1}
I20260809 03:25:14.651221 11815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.644966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:45387 (local address 127.0.0.1:55026)
0809 03:25:14.646131 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.646144 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.646340 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.646993 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.646996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.647014 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:14.647101 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.647107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.648247 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.648251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.649381 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.649390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.650479 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.650483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.650675 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.650770 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.650886 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.651036 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1106,"mutex_wait_us":92,"thread_start_us":92,"threads_started":1}
I20260809 03:25:14.651221 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.644662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:36035 (local address 127.0.0.1:45944)
0809 03:25:14.645370 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.645384 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.645534 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.646339 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.646344 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.646366 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:14.646491 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.646498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.647786 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.647789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.649467 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.649478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.650479 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.650483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.650697 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.650773 (+ 76us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.650886 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.651036 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":278,"spinlock_wait_cycles":13312,"thread_start_us":121,"threads_started":1}
I20260809 03:25:14.651690 11819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.645399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.9.46.194:36035)
0809 03:25:14.645836 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.645849 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.645873 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:25:14.645946 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.645952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.646147 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.646661 (+ 514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:14.646774 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.647695 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.649595 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.650354 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.650455 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.650624 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.651435 (+ 811us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.651550 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":131,"threads_started":1}
I20260809 03:25:14.652216 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.644610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:45377 (local address 127.0.0.1:40396)
0809 03:25:14.645305 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.645353 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.645534 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.646454 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.646457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.646471 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:14.646550 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.646558 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.647733 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.647737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.649467 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.649478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.651810 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.651813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.651909 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.651942 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.652013 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.652088 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":203,"spinlock_wait_cycles":4224,"thread_start_us":78,"threads_started":1}
I20260809 03:25:14.654022 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.644739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40396 (local address 127.9.46.195:45377)
0809 03:25:14.645836 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.645849 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.645873 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:25:14.645947 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.645952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.646147 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.646305 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:14.646674 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.647619 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.650697 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.651464 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.651582 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.651740 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.652367 (+ 627us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.652461 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":996,"thread_start_us":118,"threads_started":1}
I20260809 03:25:14.654206 11429 tablet_service.cc:1467] Processing CreateTablet for tablet 9daf71fc995c4df688887807c3a422e1 (DEFAULT_TABLE table=test-table [id=34f3310c8634477d9769c67247daac5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:14.654196 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 9daf71fc995c4df688887807c3a422e1 (DEFAULT_TABLE table=test-table [id=34f3310c8634477d9769c67247daac5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:14.655318 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9daf71fc995c4df688887807c3a422e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:14.655318 11429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9daf71fc995c4df688887807c3a422e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:14.656605 11727 tablet_service.cc:1467] Processing CreateTablet for tablet 9daf71fc995c4df688887807c3a422e1 (DEFAULT_TABLE table=test-table [id=34f3310c8634477d9769c67247daac5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:14.657598 11727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9daf71fc995c4df688887807c3a422e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:14.662182 11821 tablet_bootstrap.cc:492] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Bootstrap starting.
I20260809 03:25:14.664496 11821 tablet_bootstrap.cc:654] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:14.664862 11823 tablet_bootstrap.cc:492] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: Bootstrap starting.
I20260809 03:25:14.665390 11822 tablet_bootstrap.cc:492] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: Bootstrap starting.
I20260809 03:25:14.665380 11821 log.cc:824] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:14.666997 11823 tablet_bootstrap.cc:654] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:14.667095 11821 tablet_bootstrap.cc:492] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: No bootstrap required, opened a new log
I20260809 03:25:14.667284 11821 ts_tablet_manager.cc:1397] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 03:25:14.667794 11822 tablet_bootstrap.cc:654] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:14.667836 11823 log.cc:824] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:14.668648 11822 log.cc:824] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:14.669934 11823 tablet_bootstrap.cc:492] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: No bootstrap required, opened a new log
I20260809 03:25:14.670125 11823 ts_tablet_manager.cc:1397] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:25:14.670380 11822 tablet_bootstrap.cc:492] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: No bootstrap required, opened a new log
I20260809 03:25:14.670588 11822 ts_tablet_manager.cc:1397] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:25:14.671149 11821 raft_consensus.cc:348] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } }
I20260809 03:25:14.671418 11821 raft_consensus.cc:374] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:14.671474 11821 raft_consensus.cc:729] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a8f4d68d8048578eeee3ac1c0c3ba8, State: Initialized, Role: FOLLOWER
I20260809 03:25:14.671942 11821 consensus_queue.cc:260] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } }
I20260809 03:25:14.672674 11649 heartbeater.cc:503] Master 127.9.46.254:43101 was elected leader, sending a full tablet report...
I20260809 03:25:14.673045 11821 ts_tablet_manager.cc:1428] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260809 03:25:14.673952 11823 raft_consensus.cc:348] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } }
I20260809 03:25:14.674206 11823 raft_consensus.cc:374] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:14.674273 11823 raft_consensus.cc:729] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ff3d127ff64be29e732807d0abf08c, State: Initialized, Role: FOLLOWER
I20260809 03:25:14.674429 11822 raft_consensus.cc:348] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } }
I20260809 03:25:14.674705 11822 raft_consensus.cc:374] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:14.674770 11822 raft_consensus.cc:729] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc863088131c41cd83b6ef0f88a6faae, State: Initialized, Role: FOLLOWER
I20260809 03:25:14.674690 11823 consensus_queue.cc:260] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } }
I20260809 03:25:14.675168 11822 consensus_queue.cc:260] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } }
I20260809 03:25:14.675688 11823 ts_tablet_manager.cc:1428] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260809 03:25:14.676074 11822 ts_tablet_manager.cc:1428] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:25:14.676102 11797 heartbeater.cc:503] Master 127.9.46.254:43101 was elected leader, sending a full tablet report...
I20260809 03:25:14.676388 11503 heartbeater.cc:503] Master 127.9.46.254:43101 was elected leader, sending a full tablet report...
I20260809 03:25:14.796743 11827 raft_consensus.cc:482] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:14.797137 11827 raft_consensus.cc:504] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } }
I20260809 03:25:14.798426 11827 leader_election.cc:288] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377), bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387)
I20260809 03:25:14.802469 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.798674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:45387 (local address 127.9.46.194:48281)
0809 03:25:14.798799 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.798816 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.798914 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.799221 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.799225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.799245 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:14.799381 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.799387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.800241 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.800247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.801203 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.801213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.801917 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.801921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.802091 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.802110 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.802234 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.802318 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":60}
I20260809 03:25:14.802469 11820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.798817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:48281 (local address 127.9.46.193:45387)
0809 03:25:14.798951 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.798959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.798991 (+ 32us) server_negotiation.cc:407] Connection header received
0809 03:25:14.799059 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.799068 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.799114 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.799209 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:14.799511 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.800115 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.801343 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.801784 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.801927 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.801987 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.802268 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.802350 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:25:14.803294 11830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.798673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:45377 (local address 127.9.46.194:38183)
0809 03:25:14.799186 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.799199 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.799339 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.799653 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.799656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.799687 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:14.800237 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.800252 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.801180 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.801186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.802047 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.802057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.802914 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.802918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.803033 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.803054 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.803124 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.803176 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":136,"thread_start_us":189,"threads_started":1}
I20260809 03:25:14.803393 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.798826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:38183 (local address 127.9.46.195:45377)
0809 03:25:14.798995 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.799002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.799295 (+ 293us) server_negotiation.cc:407] Connection header received
0809 03:25:14.799480 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.799484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.799539 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.799643 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:14.800384 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.801059 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.802186 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.802766 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.802905 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.802974 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.803164 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.803246 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:25:14.803318 11455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9daf71fc995c4df688887807c3a422e1" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc863088131c41cd83b6ef0f88a6faae" is_pre_election: true
I20260809 03:25:14.803678 11455 raft_consensus.cc:2454] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 0.
I20260809 03:25:14.804105 11748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9daf71fc995c4df688887807c3a422e1" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ff3d127ff64be29e732807d0abf08c" is_pre_election: true
I20260809 03:25:14.804286 11529 leader_election.cc:302] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a8f4d68d8048578eeee3ac1c0c3ba8, bc863088131c41cd83b6ef0f88a6faae; no voters:
I20260809 03:25:14.804435 11748 raft_consensus.cc:2454] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 0.
I20260809 03:25:14.804695 11827 raft_consensus.cc:2781] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:14.804808 11827 raft_consensus.cc:482] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:14.804874 11827 raft_consensus.cc:3037] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:14.806022 11827 raft_consensus.cc:504] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } }
I20260809 03:25:14.806443 11827 leader_election.cc:288] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 election: Requested vote from peers f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377), bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387)
I20260809 03:25:14.806807 11748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9daf71fc995c4df688887807c3a422e1" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ff3d127ff64be29e732807d0abf08c"
I20260809 03:25:14.806865 11455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9daf71fc995c4df688887807c3a422e1" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc863088131c41cd83b6ef0f88a6faae"
I20260809 03:25:14.806959 11748 raft_consensus.cc:3037] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:14.807016 11455 raft_consensus.cc:3037] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:14.808132 11455 raft_consensus.cc:2454] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 1.
I20260809 03:25:14.808435 11748 raft_consensus.cc:2454] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 1.
I20260809 03:25:14.808466 11529 leader_election.cc:302] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a8f4d68d8048578eeee3ac1c0c3ba8, bc863088131c41cd83b6ef0f88a6faae; no voters:
I20260809 03:25:14.808696 11827 raft_consensus.cc:2781] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:14.808943 11827 raft_consensus.cc:686] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 LEADER]: Becoming Leader. State: Replica: 00a8f4d68d8048578eeee3ac1c0c3ba8, State: Running, Role: LEADER
I20260809 03:25:14.809322 11827 consensus_queue.cc:237] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } }
I20260809 03:25:14.812462 11287 catalog_manager.cc:5495] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194). New cstate: current_term: 1 leader_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } health_report { overall_health: UNKNOWN } } }
W20260809 03:25:14.837867 11798 tablet.cc:2369] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 03:25:14 dist-test-slave-65mx krb5kdc[11241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245913, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:14.868057 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.858591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43101 (local address 127.0.0.1:50268)
0809 03:25:14.859076 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.859111 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.859239 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.859564 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.859568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.860227 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:14.860364 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.860374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.861464 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.861469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.862034 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.862043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.862504 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.862507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.862599 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.863254 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:14.863266 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:14.864862 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:14.866436 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.866449 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.866456 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.866690 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.866970 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:14.866974 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:14.866977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:14.867061 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:14.867444 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:14.867451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:14.867573 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.867687 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.867855 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":134,"threads_started":1}
I20260809 03:25:14.868057 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.858701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50268 (local address 127.9.46.254:43101)
0809 03:25:14.858838 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.858844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.859177 (+ 333us) server_negotiation.cc:407] Connection header received
0809 03:25:14.859322 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.859325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.859365 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.859445 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:14.860524 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.861339 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.862158 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.862390 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.862500 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.864994 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:14.864998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:14.865001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:14.865023 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:14.866325 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.866791 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.866797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.866799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.866879 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:14.867149 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:14.867155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:14.867157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:14.867340 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:14.867445 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.867746 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.867907 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":51}
W20260809 03:25:14.870276 11650 tablet.cc:2369] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:14.873027 11287 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:50268:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 03:25:14.873857 11287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:14.879084 11727 tablet_service.cc:1467] Processing CreateTablet for tablet 2e8f5ecca3a54e84b98821cdc918b330 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca891afeff7a4153a45afca28efb8e45]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:25:14.879086 11429 tablet_service.cc:1467] Processing CreateTablet for tablet 2e8f5ecca3a54e84b98821cdc918b330 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca891afeff7a4153a45afca28efb8e45]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:25:14.879048 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 2e8f5ecca3a54e84b98821cdc918b330 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca891afeff7a4153a45afca28efb8e45]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:25:14.879758 11727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e8f5ecca3a54e84b98821cdc918b330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:14.879755 11429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e8f5ecca3a54e84b98821cdc918b330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:14.879755 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e8f5ecca3a54e84b98821cdc918b330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:14.882335 11823 tablet_bootstrap.cc:492] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: Bootstrap starting.
I20260809 03:25:14.882711 11821 tablet_bootstrap.cc:492] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Bootstrap starting.
I20260809 03:25:14.883368 11823 tablet_bootstrap.cc:654] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:14.883744 11821 tablet_bootstrap.cc:654] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:14.884512 11822 tablet_bootstrap.cc:492] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: Bootstrap starting.
I20260809 03:25:14.885089 11823 tablet_bootstrap.cc:492] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: No bootstrap required, opened a new log
I20260809 03:25:14.885174 11823 ts_tablet_manager.cc:1397] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 03:25:14.885515 11822 tablet_bootstrap.cc:654] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:14.885597 11823 raft_consensus.cc:348] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:14.885711 11823 raft_consensus.cc:374] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:14.885739 11823 raft_consensus.cc:729] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ff3d127ff64be29e732807d0abf08c, State: Initialized, Role: FOLLOWER
I20260809 03:25:14.885859 11823 consensus_queue.cc:260] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:14.886093 11823 ts_tablet_manager.cc:1428] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:25:14.886312 11828 tablet_replica.cc:439] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } }
I20260809 03:25:14.886467 11828 tablet_replica.cc:442] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:14.887029 11821 tablet_bootstrap.cc:492] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: No bootstrap required, opened a new log
I20260809 03:25:14.887125 11821 ts_tablet_manager.cc:1397] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 03:25:14.887609 11821 raft_consensus.cc:348] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:14.887739 11821 raft_consensus.cc:374] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:14.887776 11821 raft_consensus.cc:729] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a8f4d68d8048578eeee3ac1c0c3ba8, State: Initialized, Role: FOLLOWER
I20260809 03:25:14.887943 11822 tablet_bootstrap.cc:492] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: No bootstrap required, opened a new log
I20260809 03:25:14.887923 11821 consensus_queue.cc:260] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:14.888048 11822 ts_tablet_manager.cc:1397] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260809 03:25:14.888201 11827 tablet_replica.cc:439] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } }
I20260809 03:25:14.888327 11827 tablet_replica.cc:442] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:14.888554 11822 raft_consensus.cc:348] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:14.888715 11822 raft_consensus.cc:374] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:14.888751 11822 raft_consensus.cc:729] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc863088131c41cd83b6ef0f88a6faae, State: Initialized, Role: FOLLOWER
I20260809 03:25:14.888865 11822 consensus_queue.cc:260] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:14.889160 11822 ts_tablet_manager.cc:1428] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:25:14.889120 11829 tablet_replica.cc:439] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } }
I20260809 03:25:14.889247 11829 tablet_replica.cc:442] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:14.888931 11821 ts_tablet_manager.cc:1428] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260809 03:25:14.904353 11504 tablet.cc:2369] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:14.976737 11829 raft_consensus.cc:482] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:14.976898 11829 raft_consensus.cc:504] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:14.978113 11829 leader_election.cc:288] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194:36035), f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377)
I20260809 03:25:14.983462 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.978522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:39753 (local address 127.9.46.195:45377)
0809 03:25:14.978663 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.978668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.979108 (+ 440us) server_negotiation.cc:407] Connection header received
0809 03:25:14.979168 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.979171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.979219 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.979316 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:14.979743 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.980502 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.982097 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.982712 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.982846 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.982897 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.983260 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.983330 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:25:14.983476 11819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.978508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:38009 (local address 127.9.46.194:36035)
0809 03:25:14.978676 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:14.978680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:14.978967 (+ 287us) server_negotiation.cc:407] Connection header received
0809 03:25:14.979163 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:14.979166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:14.979223 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:14.979316 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:14.980075 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.980776 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.982098 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.982739 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.982846 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.982898 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:14.983260 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 03:25:14.983330 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 03:25:14.983548 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.978394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:36035 (local address 127.9.46.193:38009)
0809 03:25:14.978846 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.978863 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.978994 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.979344 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.979347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.979366 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:14.979947 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.979954 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.980881 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.980884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.981972 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.981983 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.982887 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.982891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.983084 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.983140 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.983219 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.983346 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":128,"spinlock_wait_cycles":3072,"thread_start_us":129,"threads_started":1}
I20260809 03:25:14.983548 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:14.978394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:45377 (local address 127.9.46.193:39753)
0809 03:25:14.978948 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:14.978960 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:14.979045 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:14.979344 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:14.979347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:14.979391 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:14.979601 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.979612 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.980626 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.980629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.981926 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:14.981938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:14.982887 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:14.982891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:14.983095 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:14.983136 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:25:14.983219 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:25:14.983353 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":222,"spinlock_wait_cycles":36480,"thread_start_us":121,"threads_started":1}
I20260809 03:25:14.984237 11600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8f5ecca3a54e84b98821cdc918b330" candidate_uuid: "bc863088131c41cd83b6ef0f88a6faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" is_pre_election: true
I20260809 03:25:14.984406 11748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8f5ecca3a54e84b98821cdc918b330" candidate_uuid: "bc863088131c41cd83b6ef0f88a6faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ff3d127ff64be29e732807d0abf08c" is_pre_election: true
I20260809 03:25:14.984526 11600 raft_consensus.cc:2454] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc863088131c41cd83b6ef0f88a6faae in term 0.
I20260809 03:25:14.984582 11748 raft_consensus.cc:2454] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc863088131c41cd83b6ef0f88a6faae in term 0.
I20260809 03:25:14.984988 11383 leader_election.cc:302] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 00a8f4d68d8048578eeee3ac1c0c3ba8, bc863088131c41cd83b6ef0f88a6faae, f9ff3d127ff64be29e732807d0abf08c; no voters:
I20260809 03:25:14.985344 11829 raft_consensus.cc:2781] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:14.985412 11829 raft_consensus.cc:482] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:14.985452 11829 raft_consensus.cc:3037] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:14.986444 11829 raft_consensus.cc:504] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:14.986779 11829 leader_election.cc:288] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [CANDIDATE]: Term 1 election: Requested vote from peers 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194:36035), f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377)
I20260809 03:25:14.987073 11600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8f5ecca3a54e84b98821cdc918b330" candidate_uuid: "bc863088131c41cd83b6ef0f88a6faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8"
I20260809 03:25:14.987110 11748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e8f5ecca3a54e84b98821cdc918b330" candidate_uuid: "bc863088131c41cd83b6ef0f88a6faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ff3d127ff64be29e732807d0abf08c"
I20260809 03:25:14.987200 11600 raft_consensus.cc:3037] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:14.987252 11748 raft_consensus.cc:3037] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:14.988137 11748 raft_consensus.cc:2454] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc863088131c41cd83b6ef0f88a6faae in term 1.
I20260809 03:25:14.988180 11600 raft_consensus.cc:2454] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc863088131c41cd83b6ef0f88a6faae in term 1.
I20260809 03:25:14.988420 11383 leader_election.cc:302] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc863088131c41cd83b6ef0f88a6faae, f9ff3d127ff64be29e732807d0abf08c; no voters:
I20260809 03:25:14.988627 11829 raft_consensus.cc:2781] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:14.988873 11829 raft_consensus.cc:686] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [term 1 LEADER]: Becoming Leader. State: Replica: bc863088131c41cd83b6ef0f88a6faae, State: Running, Role: LEADER
I20260809 03:25:14.989203 11829 consensus_queue.cc:237] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:14.991446 11846 tablet_replica.cc:439] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: TxnStatusTablet state changed. Reason: New leader bc863088131c41cd83b6ef0f88a6faae. Latest consensus state: current_term: 1 leader_uuid: "bc863088131c41cd83b6ef0f88a6faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } }
I20260809 03:25:14.991560 11846 tablet_replica.cc:442] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:25:14.992254 11848 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 03:25:14.992466 11848 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 03:25:14.992807 11287 catalog_manager.cc:5495] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae reported cstate change: term changed from 0 to 1, leader changed from <none> to bc863088131c41cd83b6ef0f88a6faae (127.9.46.193). New cstate: current_term: 1 leader_uuid: "bc863088131c41cd83b6ef0f88a6faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } health_report { overall_health: UNKNOWN } } }
I20260809 03:25:15.018672 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:15.014910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:45387 (local address 127.0.0.1:55030)
0809 03:25:15.015083 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:15.015101 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:15.015236 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:15.015562 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:15.015565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:15.015573 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:15.015688 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:15.015693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.016430 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.016435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:15.017068 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:15.017083 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.017535 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.017538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:15.017644 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:15.017687 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:15.018362 (+ 675us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:15.018379 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 03:25:15.018451 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 03:25:15.018538 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 03:25:15.018673 11820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:15.015081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55030 (local address 127.9.46.193:45387)
0809 03:25:15.015234 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:15.015239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:15.015254 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:25:15.015376 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:15.015381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:15.015441 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:15.015546 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:15.015803 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.016314 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.017200 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.017425 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.017535 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:15.017848 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:15.018258 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:15.018387 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:15.018462 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 03:25:15.018521 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:25:15.024173 11748 raft_consensus.cc:1261] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Refusing update from remote peer bc863088131c41cd83b6ef0f88a6faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:15.024204 11600 raft_consensus.cc:1261] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Refusing update from remote peer bc863088131c41cd83b6ef0f88a6faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:15.024921 11846 consensus_queue.cc:1035] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:15.025187 11829 consensus_queue.cc:1035] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:15.028578 11827 tablet_replica.cc:439] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: TxnStatusTablet state changed. Reason: New leader bc863088131c41cd83b6ef0f88a6faae. Latest consensus state: current_term: 1 leader_uuid: "bc863088131c41cd83b6ef0f88a6faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } }
I20260809 03:25:15.028707 11827 tablet_replica.cc:442] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:15.028774 11828 tablet_replica.cc:439] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: TxnStatusTablet state changed. Reason: New leader bc863088131c41cd83b6ef0f88a6faae. Latest consensus state: current_term: 1 leader_uuid: "bc863088131c41cd83b6ef0f88a6faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } }
I20260809 03:25:15.028903 11828 tablet_replica.cc:442] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:15.031545 11829 tablet_replica.cc:439] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc863088131c41cd83b6ef0f88a6faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } }
I20260809 03:25:15.031673 11829 tablet_replica.cc:442] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:25:15.032155 11829 tablet_replica.cc:439] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc863088131c41cd83b6ef0f88a6faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } }
I20260809 03:25:15.032277 11829 tablet_replica.cc:442] T 2e8f5ecca3a54e84b98821cdc918b330 P bc863088131c41cd83b6ef0f88a6faae: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:25:15.032423 11851 mvcc.cc:205] Tried to move back new op lower bound from 7316463267932065792 to 7316463267798085632. Current Snapshot: MvccSnapshot[applied={T|T < 7316463267932065792}]
I20260809 03:25:15.033321 11828 tablet_replica.cc:439] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc863088131c41cd83b6ef0f88a6faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } }
I20260809 03:25:15.033453 11828 tablet_replica.cc:442] T 2e8f5ecca3a54e84b98821cdc918b330 P f9ff3d127ff64be29e732807d0abf08c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:15.033804 11852 mvcc.cc:205] Tried to move back new op lower bound from 7316463267932065792 to 7316463267798085632. Current Snapshot: MvccSnapshot[applied={T|T < 7316463267932065792}]
I20260809 03:25:15.034973 11827 tablet_replica.cc:439] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc863088131c41cd83b6ef0f88a6faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } } }
I20260809 03:25:15.035104 11827 tablet_replica.cc:442] T 2e8f5ecca3a54e84b98821cdc918b330 P 00a8f4d68d8048578eeee3ac1c0c3ba8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:15.035375 11853 mvcc.cc:205] Tried to move back new op lower bound from 7316463267932065792 to 7316463267798085632. Current Snapshot: MvccSnapshot[applied={T|T < 7316463267932065792}]
I20260809 03:25:15.046706 11819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:15.042318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45956 (local address 127.9.46.194:36035)
0809 03:25:15.042436 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:15.042441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:15.042521 (+ 80us) server_negotiation.cc:407] Connection header received
0809 03:25:15.042727 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:15.042731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:15.042794 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:15.042895 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:15.043181 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.043966 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.045001 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.045381 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.045483 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:15.045754 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:15.046275 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:15.046393 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:15.046506 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 03:25:15.046563 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:25:15.058588 11820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:15.054756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:47139 (local address 127.9.46.193:45387)
0809 03:25:15.054897 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:15.054908 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:15.055177 (+ 269us) server_negotiation.cc:407] Connection header received
0809 03:25:15.055344 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:15.055347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:15.055406 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:15.055506 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:15.055781 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.056299 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.057361 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.057588 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.057708 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:15.058081 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:15.058221 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:15.058298 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:15.058390 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 03:25:15.058440 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:25:15.058633 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:15.054647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:45387 (local address 127.9.46.194:47139)
0809 03:25:15.055055 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:15.055068 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:15.055209 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:15.055506 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:15.055509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:15.055516 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:15.055666 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:15.055677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.056419 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.056422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:15.057234 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:15.057244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.057724 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.057727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:15.057829 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:15.057887 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:15.058305 (+ 418us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:15.058315 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 03:25:15.058379 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 03:25:15.058460 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":101,"threads_started":1}
I20260809 03:25:15.065506 11748 raft_consensus.cc:1261] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Refusing update from remote peer 00a8f4d68d8048578eeee3ac1c0c3ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:15.065651 11455 raft_consensus.cc:1261] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Refusing update from remote peer 00a8f4d68d8048578eeee3ac1c0c3ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:15.066022 11831 consensus_queue.cc:1035] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:15.066210 11827 consensus_queue.cc:1035] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:15.090216 11819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:15.086165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:58247 (local address 127.9.46.194:36035)
0809 03:25:15.086282 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:15.086287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:15.086301 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:25:15.086376 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:15.086379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:15.086450 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:15.086525 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:15.087224 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.087851 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.089018 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.089302 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.089363 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:15.089722 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:15.089806 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:15.089889 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:15.090005 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 03:25:15.090057 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:25:15.090217 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:15.085685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:36035 (local address 127.9.46.193:58247)
0809 03:25:15.086142 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:15.086161 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:15.086284 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:15.086931 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:15.086934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:15.086944 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:15.087089 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:15.087097 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.088000 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.088003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:15.088877 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:15.088888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:15.089457 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:15.089460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:15.089532 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:15.089575 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:15.089893 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:15.089902 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 03:25:15.089992 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:25:15.090080 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":158,"threads_started":1}
I20260809 03:25:15.120570 11285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:15.120810 11285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:15.123865 11285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d: Sending DeleteTablet for 3 replicas of tablet 9daf71fc995c4df688887807c3a422e1
I20260809 03:25:15.124882 11727 tablet_service.cc:1514] Processing DeleteTablet for tablet 9daf71fc995c4df688887807c3a422e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:15 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:40396
I20260809 03:25:15.125450 11871 tablet_replica.cc:330] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: stopping tablet replica
I20260809 03:25:15.125634 11429 tablet_service.cc:1514] Processing DeleteTablet for tablet 9daf71fc995c4df688887807c3a422e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:15 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:55026
I20260809 03:25:15.126029 11871 raft_consensus.cc:2229] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:15.126330 11871 raft_consensus.cc:2258] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:15.126539 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 9daf71fc995c4df688887807c3a422e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:15 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:45944
I20260809 03:25:15.127962 11871 ts_tablet_manager.cc:1905] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:15.127928 11285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:15.128437 11285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:15.131592 11871 ts_tablet_manager.cc:1918] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:15.131796 11871 log.cc:1196] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/9daf71fc995c4df688887807c3a422e1
I20260809 03:25:15.132221 11871 ts_tablet_manager.cc:1939] T 9daf71fc995c4df688887807c3a422e1 P f9ff3d127ff64be29e732807d0abf08c: Deleting consensus metadata
I20260809 03:25:15.133697 11274 catalog_manager.cc:4841] TS f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377): tablet 9daf71fc995c4df688887807c3a422e1 (table test-table [id=34f3310c8634477d9769c67247daac5f]) successfully deleted
I20260809 03:25:15.134282 11727 tablet_service.cc:1467] Processing CreateTablet for tablet 02a49e5a3869497393a0489c3d49b6ab (DEFAULT_TABLE table=test-table [id=851e4eb301354e35a8c86daa7c3df6a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:15.134539 11727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a49e5a3869497393a0489c3d49b6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:15.134742 11429 tablet_service.cc:1467] Processing CreateTablet for tablet 02a49e5a3869497393a0489c3d49b6ab (DEFAULT_TABLE table=test-table [id=851e4eb301354e35a8c86daa7c3df6a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:15.135066 11429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a49e5a3869497393a0489c3d49b6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:15.136845 11823 tablet_bootstrap.cc:492] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c: Bootstrap starting.
I20260809 03:25:15.137796 11823 tablet_bootstrap.cc:654] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:15.137881 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 02a49e5a3869497393a0489c3d49b6ab (DEFAULT_TABLE table=test-table [id=851e4eb301354e35a8c86daa7c3df6a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:15.138203 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a49e5a3869497393a0489c3d49b6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:15.139592 11823 tablet_bootstrap.cc:492] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c: No bootstrap required, opened a new log
I20260809 03:25:15.139680 11823 ts_tablet_manager.cc:1397] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:25:15.140116 11823 raft_consensus.cc:348] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:15.140223 11823 raft_consensus.cc:374] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:15.140251 11823 raft_consensus.cc:729] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ff3d127ff64be29e732807d0abf08c, State: Initialized, Role: FOLLOWER
I20260809 03:25:15.140367 11823 consensus_queue.cc:260] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:15.140632 11823 ts_tablet_manager.cc:1428] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:15.140936 11822 tablet_bootstrap.cc:492] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae: Bootstrap starting.
I20260809 03:25:15.142457 11822 tablet_bootstrap.cc:654] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:15.144117 11822 tablet_bootstrap.cc:492] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae: No bootstrap required, opened a new log
I20260809 03:25:15.144224 11822 ts_tablet_manager.cc:1397] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260809 03:25:15.145118 11821 tablet_bootstrap.cc:492] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8: Bootstrap starting.
I20260809 03:25:15.144748 11822 raft_consensus.cc:348] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:15.145853 11822 raft_consensus.cc:374] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:15.145891 11822 raft_consensus.cc:729] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc863088131c41cd83b6ef0f88a6faae, State: Initialized, Role: FOLLOWER
I20260809 03:25:15.146073 11822 consensus_queue.cc:260] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:15.146176 11821 tablet_bootstrap.cc:654] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:15.146467 11822 ts_tablet_manager.cc:1428] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:15.146716 11873 tablet_replica.cc:330] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: stopping tablet replica
I20260809 03:25:15.147099 11873 raft_consensus.cc:2229] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:15.147531 11873 raft_consensus.cc:2258] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:15.147876 11872 tablet_replica.cc:330] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: stopping tablet replica
I20260809 03:25:15.148144 11821 tablet_bootstrap.cc:492] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8: No bootstrap required, opened a new log
I20260809 03:25:15.148232 11821 ts_tablet_manager.cc:1397] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 03:25:15.148254 11872 raft_consensus.cc:2229] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:15.148417 11872 raft_consensus.cc:2258] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:15.149035 11821 raft_consensus.cc:348] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:15.149168 11821 raft_consensus.cc:374] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:15.149209 11821 raft_consensus.cc:729] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a8f4d68d8048578eeee3ac1c0c3ba8, State: Initialized, Role: FOLLOWER
I20260809 03:25:15.149327 11821 consensus_queue.cc:260] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:15.149600 11821 ts_tablet_manager.cc:1428] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:15.151615 11872 ts_tablet_manager.cc:1905] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:15.155992 11872 ts_tablet_manager.cc:1918] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:15.156114 11872 log.cc:1196] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/9daf71fc995c4df688887807c3a422e1
I20260809 03:25:15.156721 11872 ts_tablet_manager.cc:1939] T 9daf71fc995c4df688887807c3a422e1 P bc863088131c41cd83b6ef0f88a6faae: Deleting consensus metadata
I20260809 03:25:15.158061 11273 catalog_manager.cc:4841] TS bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387): tablet 9daf71fc995c4df688887807c3a422e1 (table test-table [id=34f3310c8634477d9769c67247daac5f]) successfully deleted
I20260809 03:25:15.161712 11873 ts_tablet_manager.cc:1905] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:15.166111 11873 ts_tablet_manager.cc:1918] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:15.166234 11873 log.cc:1196] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/9daf71fc995c4df688887807c3a422e1
I20260809 03:25:15.166579 11873 ts_tablet_manager.cc:1939] T 9daf71fc995c4df688887807c3a422e1 P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting consensus metadata
I20260809 03:25:15.167940 11275 catalog_manager.cc:4841] TS 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194:36035): tablet 9daf71fc995c4df688887807c3a422e1 (table test-table [id=34f3310c8634477d9769c67247daac5f]) successfully deleted
I20260809 03:25:15.346112 11865 raft_consensus.cc:482] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:15.346253 11865 raft_consensus.cc:504] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:15.346738 11865 leader_election.cc:288] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387), f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377)
I20260809 03:25:15.347190 11748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a49e5a3869497393a0489c3d49b6ab" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ff3d127ff64be29e732807d0abf08c" is_pre_election: true
I20260809 03:25:15.347239 11455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a49e5a3869497393a0489c3d49b6ab" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc863088131c41cd83b6ef0f88a6faae" is_pre_election: true
I20260809 03:25:15.347359 11748 raft_consensus.cc:2454] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 0.
I20260809 03:25:15.347426 11455 raft_consensus.cc:2454] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 0.
I20260809 03:25:15.347735 11530 leader_election.cc:302] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a8f4d68d8048578eeee3ac1c0c3ba8, f9ff3d127ff64be29e732807d0abf08c; no voters:
I20260809 03:25:15.347936 11865 raft_consensus.cc:2781] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:15.347996 11865 raft_consensus.cc:482] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:15.348042 11865 raft_consensus.cc:3037] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:15.349166 11865 raft_consensus.cc:504] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:15.349504 11865 leader_election.cc:288] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 election: Requested vote from peers bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387), f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377)
I20260809 03:25:15.349864 11455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a49e5a3869497393a0489c3d49b6ab" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc863088131c41cd83b6ef0f88a6faae"
I20260809 03:25:15.350001 11455 raft_consensus.cc:3037] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:15.349948 11748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a49e5a3869497393a0489c3d49b6ab" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ff3d127ff64be29e732807d0abf08c"
I20260809 03:25:15.350113 11748 raft_consensus.cc:3037] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:15.351205 11455 raft_consensus.cc:2454] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 1.
I20260809 03:25:15.351205 11748 raft_consensus.cc:2454] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 1.
I20260809 03:25:15.351507 11530 leader_election.cc:302] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a8f4d68d8048578eeee3ac1c0c3ba8, f9ff3d127ff64be29e732807d0abf08c; no voters:
I20260809 03:25:15.351687 11865 raft_consensus.cc:2781] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:15.351804 11865 raft_consensus.cc:686] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 LEADER]: Becoming Leader. State: Replica: 00a8f4d68d8048578eeee3ac1c0c3ba8, State: Running, Role: LEADER
I20260809 03:25:15.351948 11865 consensus_queue.cc:237] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:15.353713 11287 catalog_manager.cc:5495] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194). New cstate: current_term: 1 leader_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } health_report { overall_health: UNKNOWN } } }
I20260809 03:25:15.401679 11748 raft_consensus.cc:1261] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Refusing update from remote peer 00a8f4d68d8048578eeee3ac1c0c3ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:15.401678 11455 raft_consensus.cc:1261] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Refusing update from remote peer 00a8f4d68d8048578eeee3ac1c0c3ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:15.402073 11865 consensus_queue.cc:1035] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:15.402210 11831 consensus_queue.cc:1035] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:25:15.420961 11853 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 03:25:17.003407 11285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:17.003726 11285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:17.008220 11285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d: Sending DeleteTablet for 3 replicas of tablet 02a49e5a3869497393a0489c3d49b6ab
I20260809 03:25:17.009545 11429 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a49e5a3869497393a0489c3d49b6ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:16 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:55026
I20260809 03:25:17.009792 11727 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a49e5a3869497393a0489c3d49b6ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:16 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:40396
I20260809 03:25:17.009771 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a49e5a3869497393a0489c3d49b6ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:16 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:45944
I20260809 03:25:17.010367 11904 tablet_replica.cc:330] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae: stopping tablet replica
I20260809 03:25:17.010489 11906 tablet_replica.cc:330] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8: stopping tablet replica
I20260809 03:25:17.010708 11904 raft_consensus.cc:2229] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:17.010766 11906 raft_consensus.cc:2229] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:17.011040 11904 raft_consensus.cc:2258] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:17.011315 11905 tablet_replica.cc:330] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c: stopping tablet replica
I20260809 03:25:17.011314 11906 raft_consensus.cc:2258] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:17.011685 11905 raft_consensus.cc:2229] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:17.012048 11905 raft_consensus.cc:2258] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:17.012828 11904 ts_tablet_manager.cc:1905] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:17.012858 11906 ts_tablet_manager.cc:1905] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:17.013788 11905 ts_tablet_manager.cc:1905] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:17.019188 11906 ts_tablet_manager.cc:1918] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:25:17.019366 11906 log.cc:1196] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/02a49e5a3869497393a0489c3d49b6ab
I20260809 03:25:17.019776 11905 ts_tablet_manager.cc:1918] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:25:17.019980 11905 log.cc:1196] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/02a49e5a3869497393a0489c3d49b6ab
I20260809 03:25:17.020038 11906 ts_tablet_manager.cc:1939] T 02a49e5a3869497393a0489c3d49b6ab P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting consensus metadata
I20260809 03:25:17.020375 11904 ts_tablet_manager.cc:1918] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:25:17.020530 11904 log.cc:1196] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/02a49e5a3869497393a0489c3d49b6ab
I20260809 03:25:17.021548 11904 ts_tablet_manager.cc:1939] T 02a49e5a3869497393a0489c3d49b6ab P bc863088131c41cd83b6ef0f88a6faae: Deleting consensus metadata
I20260809 03:25:17.021618 11905 ts_tablet_manager.cc:1939] T 02a49e5a3869497393a0489c3d49b6ab P f9ff3d127ff64be29e732807d0abf08c: Deleting consensus metadata
I20260809 03:25:17.022185 11275 catalog_manager.cc:4841] TS 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194:36035): tablet 02a49e5a3869497393a0489c3d49b6ab (table test-table [id=851e4eb301354e35a8c86daa7c3df6a7]) successfully deleted
I20260809 03:25:17.023514 11273 catalog_manager.cc:4841] TS bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387): tablet 02a49e5a3869497393a0489c3d49b6ab (table test-table [id=851e4eb301354e35a8c86daa7c3df6a7]) successfully deleted
I20260809 03:25:17.023531 11274 catalog_manager.cc:4841] TS f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377): tablet 02a49e5a3869497393a0489c3d49b6ab (table test-table [id=851e4eb301354e35a8c86daa7c3df6a7]) successfully deleted
Aug 09 03:25:17 dist-test-slave-65mx krb5kdc[11241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245917, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:25:17 dist-test-slave-65mx krb5kdc[11241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245917, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:17.070690 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:17.053164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40500 (local address 127.9.46.254:43101)
0809 03:25:17.053632 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:17.053646 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:17.053755 (+ 109us) server_negotiation.cc:407] Connection header received
0809 03:25:17.054039 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:17.054042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:17.054145 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:17.054311 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:17.055795 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:17.056903 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:17.058677 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:17.059219 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:17.059479 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:17.064226 (+ 4747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:17.064230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:17.064232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:17.064295 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:17.067406 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:17.068271 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:17.068274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:17.068275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:17.068402 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:17.069089 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:17.069092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:17.069093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:17.069554 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:17.069752 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:17.070112 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 03:25:17.070400 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":218,"threads_started":1}
I20260809 03:25:17.075487 11285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:17.076350 11285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:17.085270 11429 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd061720bd5481c8b0dff819cc62f2a (DEFAULT_TABLE table=test-table [id=60ac99044adc49f7b867df70fc77741b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:17.085397 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd061720bd5481c8b0dff819cc62f2a (DEFAULT_TABLE table=test-table [id=60ac99044adc49f7b867df70fc77741b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:17.085861 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd061720bd5481c8b0dff819cc62f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:17.085875 11429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd061720bd5481c8b0dff819cc62f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:17.085906 11727 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd061720bd5481c8b0dff819cc62f2a (DEFAULT_TABLE table=test-table [id=60ac99044adc49f7b867df70fc77741b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:17.086409 11727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd061720bd5481c8b0dff819cc62f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:17.089695 11915 tablet_bootstrap.cc:492] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8: Bootstrap starting.
I20260809 03:25:17.091629 11916 tablet_bootstrap.cc:492] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae: Bootstrap starting.
I20260809 03:25:17.092031 11915 tablet_bootstrap.cc:654] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:17.093456 11917 tablet_bootstrap.cc:492] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c: Bootstrap starting.
I20260809 03:25:17.094652 11916 tablet_bootstrap.cc:654] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:17.094959 11915 tablet_bootstrap.cc:492] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8: No bootstrap required, opened a new log
I20260809 03:25:17.095134 11915 ts_tablet_manager.cc:1397] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:25:17.095654 11917 tablet_bootstrap.cc:654] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:17.096088 11915 raft_consensus.cc:348] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.096426 11915 raft_consensus.cc:374] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:17.096587 11915 raft_consensus.cc:729] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a8f4d68d8048578eeee3ac1c0c3ba8, State: Initialized, Role: FOLLOWER
I20260809 03:25:17.096985 11915 consensus_queue.cc:260] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.097736 11915 ts_tablet_manager.cc:1428] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:17.098064 11917 tablet_bootstrap.cc:492] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c: No bootstrap required, opened a new log
I20260809 03:25:17.098218 11917 ts_tablet_manager.cc:1397] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 03:25:17.098467 11916 tablet_bootstrap.cc:492] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae: No bootstrap required, opened a new log
I20260809 03:25:17.098640 11916 ts_tablet_manager.cc:1397] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 03:25:17.099282 11917 raft_consensus.cc:348] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.099488 11917 raft_consensus.cc:374] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:17.099583 11917 raft_consensus.cc:729] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ff3d127ff64be29e732807d0abf08c, State: Initialized, Role: FOLLOWER
I20260809 03:25:17.099677 11916 raft_consensus.cc:348] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.099953 11916 raft_consensus.cc:374] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:17.099915 11917 consensus_queue.cc:260] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.100054 11916 raft_consensus.cc:729] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc863088131c41cd83b6ef0f88a6faae, State: Initialized, Role: FOLLOWER
I20260809 03:25:17.100385 11916 consensus_queue.cc:260] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.100901 11917 ts_tablet_manager.cc:1428] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:17.101696 11916 ts_tablet_manager.cc:1428] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:17.184919 11881 raft_consensus.cc:482] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:17.185139 11881 raft_consensus.cc:504] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.186044 11881 leader_election.cc:288] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194:36035), f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377)
I20260809 03:25:17.186707 11600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd061720bd5481c8b0dff819cc62f2a" candidate_uuid: "bc863088131c41cd83b6ef0f88a6faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" is_pre_election: true
I20260809 03:25:17.186743 11748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd061720bd5481c8b0dff819cc62f2a" candidate_uuid: "bc863088131c41cd83b6ef0f88a6faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ff3d127ff64be29e732807d0abf08c" is_pre_election: true
I20260809 03:25:17.186956 11600 raft_consensus.cc:2454] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc863088131c41cd83b6ef0f88a6faae in term 0.
I20260809 03:25:17.186996 11748 raft_consensus.cc:2454] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc863088131c41cd83b6ef0f88a6faae in term 0.
I20260809 03:25:17.187526 11383 leader_election.cc:302] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc863088131c41cd83b6ef0f88a6faae, f9ff3d127ff64be29e732807d0abf08c; no voters:
I20260809 03:25:17.187836 11881 raft_consensus.cc:2781] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:17.187927 11881 raft_consensus.cc:482] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:17.187978 11881 raft_consensus.cc:3037] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:17.189639 11881 raft_consensus.cc:504] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.190184 11881 leader_election.cc:288] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [CANDIDATE]: Term 1 election: Requested vote from peers 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194:36035), f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377)
I20260809 03:25:17.190616 11600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd061720bd5481c8b0dff819cc62f2a" candidate_uuid: "bc863088131c41cd83b6ef0f88a6faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8"
I20260809 03:25:17.190827 11600 raft_consensus.cc:3037] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:17.190773 11748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd061720bd5481c8b0dff819cc62f2a" candidate_uuid: "bc863088131c41cd83b6ef0f88a6faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ff3d127ff64be29e732807d0abf08c"
I20260809 03:25:17.190996 11748 raft_consensus.cc:3037] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:17.192719 11600 raft_consensus.cc:2454] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc863088131c41cd83b6ef0f88a6faae in term 1.
I20260809 03:25:17.192745 11748 raft_consensus.cc:2454] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc863088131c41cd83b6ef0f88a6faae in term 1.
I20260809 03:25:17.193241 11384 leader_election.cc:302] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a8f4d68d8048578eeee3ac1c0c3ba8, bc863088131c41cd83b6ef0f88a6faae; no voters:
I20260809 03:25:17.193555 11881 raft_consensus.cc:2781] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:17.193746 11881 raft_consensus.cc:686] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 1 LEADER]: Becoming Leader. State: Replica: bc863088131c41cd83b6ef0f88a6faae, State: Running, Role: LEADER
I20260809 03:25:17.193993 11881 consensus_queue.cc:237] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.197557 11289 catalog_manager.cc:5495] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae reported cstate change: term changed from 0 to 1, leader changed from <none> to bc863088131c41cd83b6ef0f88a6faae (127.9.46.193). New cstate: current_term: 1 leader_uuid: "bc863088131c41cd83b6ef0f88a6faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } health_report { overall_health: UNKNOWN } } }
I20260809 03:25:17.238123 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:17.234644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59284 (local address 127.9.46.193:45387)
0809 03:25:17.234928 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:17.234938 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:17.234950 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:25:17.234995 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:17.234997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:17.235038 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:17.235129 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:17.235409 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:17.235949 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:17.236974 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:17.237211 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:17.237325 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:17.237636 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:17.237711 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:17.237790 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:17.237904 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 03:25:17.237958 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":100,"threads_started":1}
I20260809 03:25:17.246917 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:17.243913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:44187 (local address 127.9.46.193:45387)
0809 03:25:17.244032 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:17.244035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:17.244227 (+ 192us) server_negotiation.cc:407] Connection header received
0809 03:25:17.244328 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:17.244331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:17.244371 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:17.244438 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:17.244716 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:17.245291 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:17.246062 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:17.246272 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:17.246353 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:17.246563 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:17.246622 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:17.246676 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:17.246752 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 03:25:17.246798 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":46}
I20260809 03:25:17.246915 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:17.243859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:45387 (local address 127.9.46.193:44187)
0809 03:25:17.244111 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:17.244138 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:17.244249 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:17.244460 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:17.244462 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:17.244467 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:17.244596 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:17.244605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:17.245380 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:17.245382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:17.245964 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:17.245972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:17.246371 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:17.246373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:17.246443 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:17.246466 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:17.246694 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:17.246698 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:25:17.246746 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 03:25:17.246809 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":101,"threads_started":1}
I20260809 03:25:17.252707 11748 raft_consensus.cc:1261] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Refusing update from remote peer bc863088131c41cd83b6ef0f88a6faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:17.252830 11600 raft_consensus.cc:1261] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Refusing update from remote peer bc863088131c41cd83b6ef0f88a6faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:17.253214 11884 consensus_queue.cc:1035] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:17.253451 11889 consensus_queue.cc:1035] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:17.287232 11289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:17.287384 11289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:17.289489 11289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d: Sending DeleteTablet for 3 replicas of tablet 0fd061720bd5481c8b0dff819cc62f2a
I20260809 03:25:17.290021 11429 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd061720bd5481c8b0dff819cc62f2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:17 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:55026
I20260809 03:25:17.290233 11904 tablet_replica.cc:330] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae: stopping tablet replica
I20260809 03:25:17.290337 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd061720bd5481c8b0dff819cc62f2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:17 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:45944
I20260809 03:25:17.290372 11904 raft_consensus.cc:2229] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:17.290522 11727 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd061720bd5481c8b0dff819cc62f2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:17 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:40396
I20260809 03:25:17.290674 11904 raft_consensus.cc:2258] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:17.290752 11906 tablet_replica.cc:330] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8: stopping tablet replica
I20260809 03:25:17.290869 11906 raft_consensus.cc:2229] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:17.290978 11906 raft_consensus.cc:2258] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:17.291074 11905 tablet_replica.cc:330] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c: stopping tablet replica
I20260809 03:25:17.290992 11289 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:25:17.291191 11905 raft_consensus.cc:2229] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:17.291298 11905 raft_consensus.cc:2258] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 03:25:17.291424 11289 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:17.291807 11906 ts_tablet_manager.cc:1905] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:17.292430 11905 ts_tablet_manager.cc:1905] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:17.293421 11904 ts_tablet_manager.cc:1905] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:17.294621 11906 ts_tablet_manager.cc:1918] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:17.294714 11906 log.cc:1196] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/0fd061720bd5481c8b0dff819cc62f2a
I20260809 03:25:17.294902 11905 ts_tablet_manager.cc:1918] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:17.294970 11905 log.cc:1196] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/0fd061720bd5481c8b0dff819cc62f2a
I20260809 03:25:17.294986 11906 ts_tablet_manager.cc:1939] T 0fd061720bd5481c8b0dff819cc62f2a P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting consensus metadata
I20260809 03:25:17.295221 11905 ts_tablet_manager.cc:1939] T 0fd061720bd5481c8b0dff819cc62f2a P f9ff3d127ff64be29e732807d0abf08c: Deleting consensus metadata
I20260809 03:25:17.295743 11275 catalog_manager.cc:4841] TS 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194:36035): tablet 0fd061720bd5481c8b0dff819cc62f2a (table test-table [id=60ac99044adc49f7b867df70fc77741b]) successfully deleted
I20260809 03:25:17.295969 11904 ts_tablet_manager.cc:1918] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:17.296057 11904 log.cc:1196] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/0fd061720bd5481c8b0dff819cc62f2a
I20260809 03:25:17.296063 11274 catalog_manager.cc:4841] TS f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377): tablet 0fd061720bd5481c8b0dff819cc62f2a (table test-table [id=60ac99044adc49f7b867df70fc77741b]) successfully deleted
I20260809 03:25:17.296357 11904 ts_tablet_manager.cc:1939] T 0fd061720bd5481c8b0dff819cc62f2a P bc863088131c41cd83b6ef0f88a6faae: Deleting consensus metadata
I20260809 03:25:17.297109 11273 catalog_manager.cc:4841] TS bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387): tablet 0fd061720bd5481c8b0dff819cc62f2a (table test-table [id=60ac99044adc49f7b867df70fc77741b]) successfully deleted
I20260809 03:25:17.299715 11580 tablet_service.cc:1467] Processing CreateTablet for tablet 5bc66adb268f47d5a58e71d0b021276b (DEFAULT_TABLE table=test-table [id=fb82fe822d6947a9a708909b68e2ab97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:17.299722 11429 tablet_service.cc:1467] Processing CreateTablet for tablet 5bc66adb268f47d5a58e71d0b021276b (DEFAULT_TABLE table=test-table [id=fb82fe822d6947a9a708909b68e2ab97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:17.299811 11727 tablet_service.cc:1467] Processing CreateTablet for tablet 5bc66adb268f47d5a58e71d0b021276b (DEFAULT_TABLE table=test-table [id=fb82fe822d6947a9a708909b68e2ab97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:17.299976 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bc66adb268f47d5a58e71d0b021276b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:17.299983 11429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bc66adb268f47d5a58e71d0b021276b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:17.300027 11727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bc66adb268f47d5a58e71d0b021276b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:17.302155 11915 tablet_bootstrap.cc:492] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8: Bootstrap starting.
I20260809 03:25:17.302242 11917 tablet_bootstrap.cc:492] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c: Bootstrap starting.
I20260809 03:25:17.303083 11915 tablet_bootstrap.cc:654] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:17.303532 11917 tablet_bootstrap.cc:654] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:17.303776 11916 tablet_bootstrap.cc:492] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae: Bootstrap starting.
I20260809 03:25:17.304611 11915 tablet_bootstrap.cc:492] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8: No bootstrap required, opened a new log
I20260809 03:25:17.304669 11916 tablet_bootstrap.cc:654] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:17.304708 11915 ts_tablet_manager.cc:1397] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:25:17.305120 11915 raft_consensus.cc:348] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.305266 11915 raft_consensus.cc:374] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:17.305318 11915 raft_consensus.cc:729] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a8f4d68d8048578eeee3ac1c0c3ba8, State: Initialized, Role: FOLLOWER
I20260809 03:25:17.305460 11915 consensus_queue.cc:260] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.305770 11915 ts_tablet_manager.cc:1428] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:17.306002 11917 tablet_bootstrap.cc:492] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c: No bootstrap required, opened a new log
I20260809 03:25:17.306094 11917 ts_tablet_manager.cc:1397] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 03:25:17.306552 11917 raft_consensus.cc:348] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.306691 11917 raft_consensus.cc:374] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:17.306734 11917 raft_consensus.cc:729] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ff3d127ff64be29e732807d0abf08c, State: Initialized, Role: FOLLOWER
I20260809 03:25:17.306857 11917 consensus_queue.cc:260] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.307027 11916 tablet_bootstrap.cc:492] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae: No bootstrap required, opened a new log
I20260809 03:25:17.307102 11916 ts_tablet_manager.cc:1397] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 03:25:17.307142 11917 ts_tablet_manager.cc:1428] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:17.307502 11916 raft_consensus.cc:348] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.307616 11916 raft_consensus.cc:374] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:17.307652 11916 raft_consensus.cc:729] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc863088131c41cd83b6ef0f88a6faae, State: Initialized, Role: FOLLOWER
I20260809 03:25:17.307773 11916 consensus_queue.cc:260] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.308054 11916 ts_tablet_manager.cc:1428] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 03:25:17.372462 11897 raft_consensus.cc:482] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:17.372635 11897 raft_consensus.cc:504] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.373078 11897 leader_election.cc:288] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387), f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377)
I20260809 03:25:17.373616 11748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc66adb268f47d5a58e71d0b021276b" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ff3d127ff64be29e732807d0abf08c" is_pre_election: true
I20260809 03:25:17.373626 11455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc66adb268f47d5a58e71d0b021276b" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc863088131c41cd83b6ef0f88a6faae" is_pre_election: true
I20260809 03:25:17.373767 11455 raft_consensus.cc:2454] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 0.
I20260809 03:25:17.373891 11748 raft_consensus.cc:2454] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 0.
I20260809 03:25:17.374105 11529 leader_election.cc:302] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a8f4d68d8048578eeee3ac1c0c3ba8, bc863088131c41cd83b6ef0f88a6faae; no voters:
I20260809 03:25:17.374409 11897 raft_consensus.cc:2781] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:17.374465 11897 raft_consensus.cc:482] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:17.374521 11897 raft_consensus.cc:3037] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:17.375763 11897 raft_consensus.cc:504] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.376097 11897 leader_election.cc:288] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 election: Requested vote from peers bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387), f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377)
I20260809 03:25:17.376413 11455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc66adb268f47d5a58e71d0b021276b" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc863088131c41cd83b6ef0f88a6faae"
I20260809 03:25:17.376529 11455 raft_consensus.cc:3037] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:17.376477 11748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bc66adb268f47d5a58e71d0b021276b" candidate_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9ff3d127ff64be29e732807d0abf08c"
I20260809 03:25:17.376649 11748 raft_consensus.cc:3037] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:17.377702 11455 raft_consensus.cc:2454] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 1.
I20260809 03:25:17.377774 11748 raft_consensus.cc:2454] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00a8f4d68d8048578eeee3ac1c0c3ba8 in term 1.
I20260809 03:25:17.378005 11529 leader_election.cc:302] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a8f4d68d8048578eeee3ac1c0c3ba8, bc863088131c41cd83b6ef0f88a6faae; no voters:
I20260809 03:25:17.378218 11897 raft_consensus.cc:2781] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:17.378319 11897 raft_consensus.cc:686] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 LEADER]: Becoming Leader. State: Replica: 00a8f4d68d8048578eeee3ac1c0c3ba8, State: Running, Role: LEADER
I20260809 03:25:17.378441 11897 consensus_queue.cc:237] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } }
I20260809 03:25:17.380100 11289 catalog_manager.cc:5495] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194). New cstate: current_term: 1 leader_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00a8f4d68d8048578eeee3ac1c0c3ba8" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 } health_report { overall_health: UNKNOWN } } }
I20260809 03:25:17.395097 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:17.391381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57298 (local address 127.9.46.194:36035)
0809 03:25:17.391788 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:17.391822 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:17.391833 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:25:17.391869 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:17.391871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:17.391942 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:17.392044 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:17.392315 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:17.393074 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:17.393997 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:17.394240 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:17.394349 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:17.394605 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:17.394691 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:17.394798 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:17.394892 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 03:25:17.394943 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":134,"threads_started":1}
I20260809 03:25:17.402246 11747 raft_consensus.cc:1261] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Refusing update from remote peer 00a8f4d68d8048578eeee3ac1c0c3ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:17.402346 11455 raft_consensus.cc:1261] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Refusing update from remote peer 00a8f4d68d8048578eeee3ac1c0c3ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:17.402709 11897 consensus_queue.cc:1035] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9ff3d127ff64be29e732807d0abf08c" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 45377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:17.402854 11894 consensus_queue.cc:1035] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc863088131c41cd83b6ef0f88a6faae" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 45387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:25:17.422871 11886 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 03:25:18.985858 11289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:18.986044 11289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:18.988593 11289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7136bfe290754f8f99488e8fd223f73d: Sending DeleteTablet for 3 replicas of tablet 5bc66adb268f47d5a58e71d0b021276b
I20260809 03:25:18.989217 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bc66adb268f47d5a58e71d0b021276b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:18 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:45944
I20260809 03:25:18.989202 11429 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bc66adb268f47d5a58e71d0b021276b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:18 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:55026
I20260809 03:25:18.989326 11727 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bc66adb268f47d5a58e71d0b021276b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:18 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:40396
I20260809 03:25:18.989584 11967 tablet_replica.cc:330] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae: stopping tablet replica
I20260809 03:25:18.989679 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 11366
I20260809 03:25:18.989728 11967 raft_consensus.cc:2229] T 5bc66adb268f47d5a58e71d0b021276b P bc863088131c41cd83b6ef0f88a6faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:18.991458 11966 tablet_replica.cc:330] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8: stopping tablet replica
I20260809 03:25:18.991626 11966 raft_consensus.cc:2229] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:18.991860 11966 raft_consensus.cc:2258] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:18.992288 11968 tablet_replica.cc:330] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c: stopping tablet replica
I20260809 03:25:18.992465 11968 raft_consensus.cc:2229] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:18.992642 11968 raft_consensus.cc:2258] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:18.993227 11966 ts_tablet_manager.cc:1905] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:18.993257 11968 ts_tablet_manager.cc:1905] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:18.995425 11968 ts_tablet_manager.cc:1918] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:25:18.995524 11968 log.cc:1196] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/5bc66adb268f47d5a58e71d0b021276b
I20260809 03:25:18.995784 11968 ts_tablet_manager.cc:1939] T 5bc66adb268f47d5a58e71d0b021276b P f9ff3d127ff64be29e732807d0abf08c: Deleting consensus metadata
I20260809 03:25:18.996519 11274 catalog_manager.cc:4841] TS f9ff3d127ff64be29e732807d0abf08c (127.9.46.195:45377): tablet 5bc66adb268f47d5a58e71d0b021276b (table test-table [id=fb82fe822d6947a9a708909b68e2ab97]) successfully deleted
I20260809 03:25:18.996819 11966 ts_tablet_manager.cc:1918] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 03:25:18.996912 11966 log.cc:1196] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/5bc66adb268f47d5a58e71d0b021276b
I20260809 03:25:18.997218 11966 ts_tablet_manager.cc:1939] T 5bc66adb268f47d5a58e71d0b021276b P 00a8f4d68d8048578eeee3ac1c0c3ba8: Deleting consensus metadata
I20260809 03:25:18.997996 11275 catalog_manager.cc:4841] TS 00a8f4d68d8048578eeee3ac1c0c3ba8 (127.9.46.194:36035): tablet 5bc66adb268f47d5a58e71d0b021276b (table test-table [id=fb82fe822d6947a9a708909b68e2ab97]) successfully deleted
W20260809 03:25:19.001549 11273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:45387 (error 108)
I20260809 03:25:19.001799 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 11513
I20260809 03:25:19.004428 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:19.003951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:45387 (local address 127.0.0.1:59288)
0809 03:25:19.004217 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:19.004299 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:45387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":93,"threads_started":1}
W20260809 03:25:19.004575 11273 catalog_manager.cc:4584] TS bc863088131c41cd83b6ef0f88a6faae (127.9.46.193:45387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5bc66adb268f47d5a58e71d0b021276b: Network error: Client connection negotiation failed: client connection to 127.9.46.193:45387: connect: Connection refused (error 111)
I20260809 03:25:19.010632 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 11658
I20260809 03:25:19.017722 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 11256
2026-08-09T03:25:19Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6556 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:19 dist-test-slave-65mx krb5kdc[11973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:19 dist-test-slave-65mx krb5kdc[11973](info): set up 2 sockets
Aug 09 03:25:19 dist-test-slave-65mx krb5kdc[11973](info): commencing operation
krb5kdc: starting...
W20260809 03:25:20.101150 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:20 dist-test-slave-65mx krb5kdc[11973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:20.229323 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34415
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:36463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:34415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:20.341331 11986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:20.341651 11986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:20.341727 11986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:20.345862 11986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:20.345953 11986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:20.345994 11986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:20.346055 11986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:20.346095 11986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:20.351466 11986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:36463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:34415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:20.352638 11986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:20.353678 11986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:25:20.362726 11986 server_base.cc:1034] running on GCE node
W20260809 03:25:20.362779 11991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:20.362914 11994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:20.362707 11992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:20.363524 11986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:20.364785 11986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:20.365993 11986 hybrid_clock.cc:648] HybridClock initialized: now 1786245920365973 us; error 32 us; skew 500 ppm
Aug 09 03:25:20 dist-test-slave-65mx krb5kdc[11973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245920, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:20.368733 11986 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:20.369560 11986 webserver.cc:460] Webserver started at http://127.9.46.254:44183/ using document root <none> and password file <none>
I20260809 03:25:20.370119 11986 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:20.370210 11986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:20.370445 11986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:20.372434 11986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "8c548e3f562a451986b1244501e73ea4"
format_stamp: "Formatted at 2026-08-09 03:25:20 on dist-test-slave-65mx"
server_key: "594244da12d687ad6fc12f6edc3881fd"
server_key_iv: "a6cbf8b7107c547b1267cb1114da9848"
server_key_version: "encryptionkey@0"
I20260809 03:25:20.373073 11986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "8c548e3f562a451986b1244501e73ea4"
format_stamp: "Formatted at 2026-08-09 03:25:20 on dist-test-slave-65mx"
server_key: "594244da12d687ad6fc12f6edc3881fd"
server_key_iv: "a6cbf8b7107c547b1267cb1114da9848"
server_key_version: "encryptionkey@0"
I20260809 03:25:20.376791 11986 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:25:20.379175 12001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:20.380368 11986 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:20.380509 11986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "8c548e3f562a451986b1244501e73ea4"
format_stamp: "Formatted at 2026-08-09 03:25:20 on dist-test-slave-65mx"
server_key: "594244da12d687ad6fc12f6edc3881fd"
server_key_iv: "a6cbf8b7107c547b1267cb1114da9848"
server_key_version: "encryptionkey@0"
I20260809 03:25:20.380705 11986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:20.391784 11986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:20.394559 11986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:20.394783 11986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:20.402107 11986 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:34415
I20260809 03:25:20.402136 12058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:34415 every 8 connection(s)
I20260809 03:25:20.403015 11986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:20.405666 12059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:20.406062 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 11986
I20260809 03:25:20.406216 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:20.406539 9403 external_mini_cluster.cc:1428] Setting key 73686ef038fcad8745eb0544f612abd7
I20260809 03:25:20.412149 12059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4: Bootstrap starting.
I20260809 03:25:20.414532 12059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4: Neither blocks nor log segments found. Creating new log.
Aug 09 03:25:20 dist-test-slave-65mx krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:20.415325 12059 log.cc:824] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:20.417229 12059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4: No bootstrap required, opened a new log
I20260809 03:25:20.420252 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:20.407920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54112 (local address 127.9.46.254:34415)
0809 03:25:20.408346 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:20.408362 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:20.408396 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:25:20.409125 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:20.409142 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:20.409452 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:20.409799 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:20.410682 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:20.411649 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:20.412536 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:20.413026 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:20.413206 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:20.415720 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:20.415725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:20.415736 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:20.415763 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:20.417567 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:20.418005 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:20.418011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:20.418016 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:20.418129 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:20.418430 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:20.418433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:20.418435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:20.418803 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:20.419003 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:20.419305 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 03:25:20.419575 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":141,"threads_started":1}
I20260809 03:25:20.420828 12059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c548e3f562a451986b1244501e73ea4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34415 } }
I20260809 03:25:20.421063 12059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:20.421162 12059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c548e3f562a451986b1244501e73ea4, State: Initialized, Role: FOLLOWER
I20260809 03:25:20.421605 12059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c548e3f562a451986b1244501e73ea4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34415 } }
I20260809 03:25:20.421734 12059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:20.421816 12059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:20.421938 12059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:20.422919 12059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c548e3f562a451986b1244501e73ea4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34415 } }
I20260809 03:25:20.423244 12059 leader_election.cc:302] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c548e3f562a451986b1244501e73ea4; no voters:
I20260809 03:25:20.423540 12059 leader_election.cc:288] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:20.423666 12064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:20.423899 12064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [term 1 LEADER]: Becoming Leader. State: Replica: 8c548e3f562a451986b1244501e73ea4, State: Running, Role: LEADER
I20260809 03:25:20.424283 12064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c548e3f562a451986b1244501e73ea4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34415 } }
I20260809 03:25:20.424865 12059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:20.426126 12067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c548e3f562a451986b1244501e73ea4. Latest consensus state: current_term: 1 leader_uuid: "8c548e3f562a451986b1244501e73ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c548e3f562a451986b1244501e73ea4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34415 } } }
I20260809 03:25:20.426126 12066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c548e3f562a451986b1244501e73ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c548e3f562a451986b1244501e73ea4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34415 } } }
I20260809 03:25:20.426304 12067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:20.426342 12066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:20.427289 12074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:20.429912 12074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:20.435281 12074 catalog_manager.cc:1347] Generated new cluster ID: 7b72f28387bd4f599ac022018d1d77f7
I20260809 03:25:20.435397 12074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:20.467490 12074 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:20.468719 12074 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:20.477748 12074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c548e3f562a451986b1244501e73ea4: Generated new TSK 0
I20260809 03:25:20.478451 12074 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:25:20.507679 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34415
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:36463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:20.622047 12086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:20.622332 12086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:20.622376 12086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:20.626734 12086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:20.626803 12086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:20.626864 12086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:20.632866 12086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:36463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:20.634048 12086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:20.635058 12086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:20.644869 12092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:20.644889 12095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:20.645017 12086 server_base.cc:1034] running on GCE node
W20260809 03:25:20.644906 12093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:20.645646 12086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:20.646464 12086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:20.647651 12086 hybrid_clock.cc:648] HybridClock initialized: now 1786245920647619 us; error 45 us; skew 500 ppm
Aug 09 03:25:20 dist-test-slave-65mx krb5kdc[11973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245920, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:20.650496 12086 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:25:20.651373 12086 webserver.cc:460] Webserver started at http://127.9.46.193:42433/ using document root <none> and password file <none>
I20260809 03:25:20.651940 12086 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:20.652035 12086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:20.652269 12086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:20.654395 12086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "6c7752d8a4ec4a4795c96fd60cba3c91"
format_stamp: "Formatted at 2026-08-09 03:25:20 on dist-test-slave-65mx"
server_key: "197d5b789f32974daa76debc598468ce"
server_key_iv: "1fd9ce33034302e301f615b8b4aaf407"
server_key_version: "encryptionkey@0"
I20260809 03:25:20.654990 12086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "6c7752d8a4ec4a4795c96fd60cba3c91"
format_stamp: "Formatted at 2026-08-09 03:25:20 on dist-test-slave-65mx"
server_key: "197d5b789f32974daa76debc598468ce"
server_key_iv: "1fd9ce33034302e301f615b8b4aaf407"
server_key_version: "encryptionkey@0"
I20260809 03:25:20.658855 12086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:25:20.661281 12102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:20.662461 12086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:20.662599 12086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "6c7752d8a4ec4a4795c96fd60cba3c91"
format_stamp: "Formatted at 2026-08-09 03:25:20 on dist-test-slave-65mx"
server_key: "197d5b789f32974daa76debc598468ce"
server_key_iv: "1fd9ce33034302e301f615b8b4aaf407"
server_key_version: "encryptionkey@0"
I20260809 03:25:20.662724 12086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:20.705031 12086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:20.708024 12086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:20.708253 12086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:20.708922 12086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:20.709800 12086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:20.709873 12086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:20.709949 12086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:20.709993 12086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:20.719574 12086 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:36961
I20260809 03:25:20.719604 12223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:36961 every 8 connection(s)
I20260809 03:25:20.720487 12086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:20.724965 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 12086
I20260809 03:25:20.725083 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:20.725364 9403 external_mini_cluster.cc:1428] Setting key 33577152b518bd67805cf49673ae42e4
Aug 09 03:25:20 dist-test-slave-65mx krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245920, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:25:20.737146 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:20.722380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:50155 (local address 127.9.46.254:34415)
0809 03:25:20.722534 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:20.722538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:20.723575 (+ 1037us) server_negotiation.cc:407] Connection header received
0809 03:25:20.724629 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:20.724633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:20.724682 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:20.724756 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:20.727037 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:20.727802 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:20.729211 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:20.729446 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:20.729527 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:20.732761 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:20.732764 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:20.732766 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:20.732792 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:20.734400 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:20.735072 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:20.735074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:20.735075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:20.735158 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:20.735573 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:20.735575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:20.735576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:20.735839 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:20.735956 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:20.736807 (+ 851us) server_negotiation.cc:299] Negotiation successful
0809 03:25:20.736959 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:25:20.738121 12226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:20.722751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34415 (local address 127.9.46.193:50155)
0809 03:25:20.723424 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:20.723466 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:20.724349 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:20.725745 (+ 1396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:20.725754 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:20.726233 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:20.726869 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:20.726891 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:20.727936 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:20.727939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:20.729043 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:20.729057 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:20.729620 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:20.729626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:20.729864 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:20.730602 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:20.730629 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:20.732534 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:20.734524 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:20.734533 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:20.734555 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:20.734967 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:20.735257 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:20.735260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:20.735263 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:20.735472 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:20.735994 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:20.736039 (+ 45us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:20.736457 (+ 418us) client_negotiation.cc:769] Sending connection context
0809 03:25:20.736725 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 03:25:20.737103 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":115,"threads_started":1}
I20260809 03:25:20.739495 12224 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
I20260809 03:25:20.739779 12224 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:20.740304 12224 heartbeater.cc:511] Master 127.9.46.254:34415 requested a full tablet report, sending...
I20260809 03:25:20.742071 12019 ts_manager.cc:194] Registered new tserver with Master: 6c7752d8a4ec4a4795c96fd60cba3c91 (127.9.46.193:36961)
I20260809 03:25:20.743424 12019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:50155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:25:20.757102 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34415
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:36463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:20.869252 12229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:20.869509 12229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:20.869589 12229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:20.873612 12229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:20.873672 12229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:20.873762 12229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:20.879048 12229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:36463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:20.880115 12229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:20.881152 12229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:20.891703 12238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:20.891721 12235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:20.891712 12236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:20.891902 12229 server_base.cc:1034] running on GCE node
I20260809 03:25:20.892287 12229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:20.893002 12229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:20.894192 12229 hybrid_clock.cc:648] HybridClock initialized: now 1786245920894173 us; error 41 us; skew 500 ppm
Aug 09 03:25:20 dist-test-slave-65mx krb5kdc[11973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245920, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:20.896900 12229 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:25:20.897779 12229 webserver.cc:460] Webserver started at http://127.9.46.194:40043/ using document root <none> and password file <none>
I20260809 03:25:20.898351 12229 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:20.898447 12229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:20.898685 12229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:20.900786 12229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "01a3c19c91a14783b92c65f1aab73816"
format_stamp: "Formatted at 2026-08-09 03:25:20 on dist-test-slave-65mx"
server_key: "7bed3b9c9cca1a28038e05d70ea444a9"
server_key_iv: "4dac2be688b61efa037d82320542f58e"
server_key_version: "encryptionkey@0"
I20260809 03:25:20.901358 12229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "01a3c19c91a14783b92c65f1aab73816"
format_stamp: "Formatted at 2026-08-09 03:25:20 on dist-test-slave-65mx"
server_key: "7bed3b9c9cca1a28038e05d70ea444a9"
server_key_iv: "4dac2be688b61efa037d82320542f58e"
server_key_version: "encryptionkey@0"
I20260809 03:25:20.905074 12229 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:20.907447 12245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:20.908813 12229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:20.908973 12229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "01a3c19c91a14783b92c65f1aab73816"
format_stamp: "Formatted at 2026-08-09 03:25:20 on dist-test-slave-65mx"
server_key: "7bed3b9c9cca1a28038e05d70ea444a9"
server_key_iv: "4dac2be688b61efa037d82320542f58e"
server_key_version: "encryptionkey@0"
I20260809 03:25:20.909098 12229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:20.930810 12229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:20.933961 12229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:20.934178 12229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:20.934857 12229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:20.935747 12229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:20.935819 12229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:20.935894 12229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:20.935952 12229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:20.945741 12229 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:39311
I20260809 03:25:20.945766 12364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:39311 every 8 connection(s)
I20260809 03:25:20.946713 12229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:20.954528 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 12229
I20260809 03:25:20.954665 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:20.955013 9403 external_mini_cluster.cc:1428] Setting key 51c711b6b6e0300229a42ffd248e6e83
Aug 09 03:25:20 dist-test-slave-65mx krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245920, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:20.961061 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:20.948634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:51083 (local address 127.9.46.254:34415)
0809 03:25:20.948761 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:20.948764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:20.949839 (+ 1075us) server_negotiation.cc:407] Connection header received
0809 03:25:20.950743 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:20.950746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:20.950805 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:20.950919 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:20.952084 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:20.952676 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:20.953809 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:20.954034 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:20.954143 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:20.957028 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:20.957035 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:20.957041 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:20.957073 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:20.958769 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:20.959441 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:20.959443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:20.959445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:20.959517 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:20.959920 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:20.959922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:20.959923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:20.960094 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:20.960184 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:20.960764 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 03:25:20.960891 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:25:20.961874 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:20.949018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34415 (local address 127.9.46.194:51083)
0809 03:25:20.949677 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:20.949723 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:20.950536 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:20.951096 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:20.951103 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:20.951463 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:20.951929 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:20.951942 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:20.952801 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:20.952805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:20.953676 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:20.953685 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:20.954202 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:20.954207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:20.954440 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:20.955109 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:20.955138 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:20.956841 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:20.958902 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:20.958911 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:20.958929 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:20.959342 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:20.959613 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:20.959618 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:20.959621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:20.959821 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:20.960205 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:20.960210 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:20.960472 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 03:25:20.960713 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 03:25:20.961035 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":104,"threads_started":1}
I20260809 03:25:20.963136 12365 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34415
I20260809 03:25:20.963374 12365 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:20.963855 12365 heartbeater.cc:511] Master 127.9.46.254:34415 requested a full tablet report, sending...
I20260809 03:25:20.964931 12019 ts_manager.cc:194] Registered new tserver with Master: 01a3c19c91a14783b92c65f1aab73816 (127.9.46.194:39311)
I20260809 03:25:20.965504 12019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:51083
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:25:20.985289 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34415
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:36463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:21.096787 12371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:21.097088 12371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:21.097177 12371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:21.101226 12371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:21.101438 12371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:21.101625 12371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:21.107043 12371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:36463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:21.108079 12371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:21.109177 12371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:21.118525 12377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:21.118544 12376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:21.118525 12379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:21.118813 12371 server_base.cc:1034] running on GCE node
I20260809 03:25:21.119184 12371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:21.119812 12371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:21.120997 12371 hybrid_clock.cc:648] HybridClock initialized: now 1786245921120986 us; error 36 us; skew 500 ppm
Aug 09 03:25:21 dist-test-slave-65mx krb5kdc[11973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245921, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:21.123708 12371 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:25:21.124691 12371 webserver.cc:460] Webserver started at http://127.9.46.195:42365/ using document root <none> and password file <none>
I20260809 03:25:21.125236 12371 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:21.125326 12371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:21.125586 12371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:21.127624 12371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "1b7163c511d44bcfb3f914d859932232"
format_stamp: "Formatted at 2026-08-09 03:25:21 on dist-test-slave-65mx"
server_key: "1721f8623ec93ff0f06c31fd0d14fa6e"
server_key_iv: "0eeb8b7d7120ac204bba623cf6f8f4e0"
server_key_version: "encryptionkey@0"
I20260809 03:25:21.128192 12371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "1b7163c511d44bcfb3f914d859932232"
format_stamp: "Formatted at 2026-08-09 03:25:21 on dist-test-slave-65mx"
server_key: "1721f8623ec93ff0f06c31fd0d14fa6e"
server_key_iv: "0eeb8b7d7120ac204bba623cf6f8f4e0"
server_key_version: "encryptionkey@0"
I20260809 03:25:21.131927 12371 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 03:25:21.134457 12386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:21.135751 12371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:21.135895 12371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "1b7163c511d44bcfb3f914d859932232"
format_stamp: "Formatted at 2026-08-09 03:25:21 on dist-test-slave-65mx"
server_key: "1721f8623ec93ff0f06c31fd0d14fa6e"
server_key_iv: "0eeb8b7d7120ac204bba623cf6f8f4e0"
server_key_version: "encryptionkey@0"
I20260809 03:25:21.136015 12371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:21.149621 12371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:21.153025 12371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:21.153239 12371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:21.153834 12371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:21.154664 12371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:21.154735 12371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:21.154807 12371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:21.154860 12371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:21.164595 12371 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:35629
I20260809 03:25:21.164616 12506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:35629 every 8 connection(s)
I20260809 03:25:21.165601 12371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:21.172272 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 12371
I20260809 03:25:21.172389 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:21.172712 9403 external_mini_cluster.cc:1428] Setting key 3d0bd24814e315dada461bd7273ed044
Aug 09 03:25:21 dist-test-slave-65mx krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245921, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:21.179392 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:21.167600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:48557 (local address 127.9.46.254:34415)
0809 03:25:21.167746 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:21.167750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:21.168861 (+ 1111us) server_negotiation.cc:407] Connection header received
0809 03:25:21.169705 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:21.169708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:21.169752 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:21.169823 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:21.170939 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:21.171459 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:21.172901 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:21.173193 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:21.173307 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:21.176020 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:21.176022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:21.176024 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:21.176049 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:21.177665 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:21.178123 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:21.178125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:21.178127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:21.178199 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:21.178496 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:21.178498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:21.178499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:21.178670 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:21.178745 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:21.179181 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 03:25:21.179276 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:25:21.180195 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:21.167940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34415 (local address 127.9.46.195:48557)
0809 03:25:21.168670 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:21.168714 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:21.169520 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:21.169982 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:21.169989 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:21.170337 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:21.170813 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:21.170825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:21.171565 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:21.171567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:21.172757 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:21.172769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:21.173354 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:21.173358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:21.173587 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:21.174313 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:21.174329 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:21.175866 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:21.177768 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:21.177773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:21.177783 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:21.178027 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:21.178280 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:21.178282 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:21.178283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:21.178415 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:21.178757 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:21.178762 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:21.178983 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 03:25:21.179139 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 03:25:21.179381 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":120,"threads_started":1}
I20260809 03:25:21.181305 12507 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34415
I20260809 03:25:21.181516 12507 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:21.181960 12507 heartbeater.cc:511] Master 127.9.46.254:34415 requested a full tablet report, sending...
I20260809 03:25:21.182937 12019 ts_manager.cc:194] Registered new tserver with Master: 1b7163c511d44bcfb3f914d859932232 (127.9.46.195:35629)
I20260809 03:25:21.183564 12019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:48557
I20260809 03:25:21.186678 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 03:25:21.201689 12062 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:21.200303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54128 (local address 127.9.46.254:34415)
0809 03:25:21.200447 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:21.200450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:21.200624 (+ 174us) server_negotiation.cc:407] Connection header received
0809 03:25:21.200741 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:21.200744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:21.200787 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:21.200865 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:21.201567 (+ 702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54128: BlockingRecv error: recv got EOF from 127.0.0.1:54128 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:25:21.202739 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 12086
I20260809 03:25:21.211241 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 12229
I20260809 03:25:21.221279 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 12371
I20260809 03:25:21.229131 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 11986
2026-08-09T03:25:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2210 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:21 dist-test-slave-65mx krb5kdc[12522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:21 dist-test-slave-65mx krb5kdc[12522](info): set up 2 sockets
Aug 09 03:25:21 dist-test-slave-65mx krb5kdc[12522](info): commencing operation
krb5kdc: starting...
W20260809 03:25:22.312912 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:22 dist-test-slave-65mx krb5kdc[12522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:22.439851 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:35565
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:35565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:22.573503 12536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:22.573840 12536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:22.573918 12536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:22.578066 12536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:22.578147 12536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:22.578192 12536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:22.578243 12536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:22.578280 12536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:22.583629 12536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:35565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:35565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:22.584774 12536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:22.585810 12536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:22.594532 12542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:22.594580 12541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:22.594606 12544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:22.594708 12536 server_base.cc:1034] running on GCE node
I20260809 03:25:22.595338 12536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:22.596447 12536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:22.597645 12536 hybrid_clock.cc:648] HybridClock initialized: now 1786245922597615 us; error 45 us; skew 500 ppm
Aug 09 03:25:22 dist-test-slave-65mx krb5kdc[12522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245922, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:22.600373 12536 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:22.601538 12536 webserver.cc:460] Webserver started at http://127.9.46.254:39237/ using document root <none> and password file <none>
I20260809 03:25:22.602317 12536 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:22.602416 12536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:22.602667 12536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:22.605502 12536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "7d66866fca0e42d2a38fbfc89d63ff7c"
format_stamp: "Formatted at 2026-08-09 03:25:22 on dist-test-slave-65mx"
server_key: "5390b58397cbdd878710a57288d80a2b"
server_key_iv: "88145b3b73f545a42c552c1ca9e95676"
server_key_version: "encryptionkey@0"
I20260809 03:25:22.606307 12536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "7d66866fca0e42d2a38fbfc89d63ff7c"
format_stamp: "Formatted at 2026-08-09 03:25:22 on dist-test-slave-65mx"
server_key: "5390b58397cbdd878710a57288d80a2b"
server_key_iv: "88145b3b73f545a42c552c1ca9e95676"
server_key_version: "encryptionkey@0"
I20260809 03:25:22.610427 12536 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:25:22.612985 12551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:22.614253 12536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:25:22.614401 12536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "7d66866fca0e42d2a38fbfc89d63ff7c"
format_stamp: "Formatted at 2026-08-09 03:25:22 on dist-test-slave-65mx"
server_key: "5390b58397cbdd878710a57288d80a2b"
server_key_iv: "88145b3b73f545a42c552c1ca9e95676"
server_key_version: "encryptionkey@0"
I20260809 03:25:22.614516 12536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:22.626562 12536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:22.630604 12536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:22.630828 12536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:22.638319 12536 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:35565
I20260809 03:25:22.638680 12605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:35565 every 8 connection(s)
I20260809 03:25:22.639438 12536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:22.642316 12606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:22.647135 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 12536
I20260809 03:25:22.647236 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:22.647508 9403 external_mini_cluster.cc:1428] Setting key 79ba9fa9bde1f7adad3a8f58a2f22001
I20260809 03:25:22.648319 12606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c: Bootstrap starting.
I20260809 03:25:22.650733 12606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:22.651767 12606 log.cc:824] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:22.654179 12606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c: No bootstrap required, opened a new log
Aug 09 03:25:22 dist-test-slave-65mx krb5kdc[12522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:22.658845 12606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d66866fca0e42d2a38fbfc89d63ff7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 35565 } }
I20260809 03:25:22.659155 12606 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:22.659238 12606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d66866fca0e42d2a38fbfc89d63ff7c, State: Initialized, Role: FOLLOWER
I20260809 03:25:22.659772 12606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d66866fca0e42d2a38fbfc89d63ff7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 35565 } }
I20260809 03:25:22.659942 12606 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:22.660017 12606 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:22.660126 12606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:22.662817 12609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:22.648997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.9.46.254:35565)
0809 03:25:22.649437 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:22.649453 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:22.649489 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:25:22.650153 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:22.650170 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:22.650496 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:22.650783 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:22.651800 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:22.653111 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:22.654367 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:22.654835 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:22.654957 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:22.657872 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:22.657878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:22.657891 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:22.657920 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:22.659805 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:22.660341 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:22.660345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:22.660350 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:22.660478 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:22.660803 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:22.660807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:22.660809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:22.661240 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:22.661456 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:22.661769 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 03:25:22.662068 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":130,"threads_started":1}
I20260809 03:25:22.666380 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:25:22.684047 12609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:22.679267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52290 (local address 127.9.46.254:35565)
0809 03:25:22.679419 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:22.679424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:22.679565 (+ 141us) server_negotiation.cc:407] Connection header received
0809 03:25:22.679689 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:22.679692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:22.679738 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:22.679811 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:22.680318 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:22.680910 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:22.681738 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:22.681979 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:22.682056 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:22.682966 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:22.682969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:22.682971 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:22.683474 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:22.683504 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:22.683519 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:22.683595 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:22.683720 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 03:25:22.683773 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:25:22.705257 12606 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d66866fca0e42d2a38fbfc89d63ff7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 35565 } }
I20260809 03:25:22.705786 12606 leader_election.cc:302] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d66866fca0e42d2a38fbfc89d63ff7c; no voters:
I20260809 03:25:22.706176 12606 leader_election.cc:288] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:22.706502 12618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:22.707093 12618 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [term 1 LEADER]: Becoming Leader. State: Replica: 7d66866fca0e42d2a38fbfc89d63ff7c, State: Running, Role: LEADER
I20260809 03:25:22.707436 12618 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d66866fca0e42d2a38fbfc89d63ff7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 35565 } }
I20260809 03:25:22.707747 12606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:22.709236 12619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d66866fca0e42d2a38fbfc89d63ff7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d66866fca0e42d2a38fbfc89d63ff7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 35565 } } }
I20260809 03:25:22.709371 12619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:22.709831 12621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d66866fca0e42d2a38fbfc89d63ff7c. Latest consensus state: current_term: 1 leader_uuid: "7d66866fca0e42d2a38fbfc89d63ff7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d66866fca0e42d2a38fbfc89d63ff7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 35565 } } }
I20260809 03:25:22.710031 12621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:22.710357 12630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:22.713295 12630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:22.719172 12630 catalog_manager.cc:1347] Generated new cluster ID: a9a58bc4078f4d7580e7a1feb598c17f
I20260809 03:25:22.719285 12630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:22.734205 12630 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:22.735677 12630 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:22.745002 12630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d66866fca0e42d2a38fbfc89d63ff7c: Generated new TSK 0
I20260809 03:25:22.745885 12630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:25:22.754029 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 12536
2026-08-09T03:25:22Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1524 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:24 dist-test-slave-65mx krb5kdc[12643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:24 dist-test-slave-65mx krb5kdc[12643](info): set up 2 sockets
Aug 09 03:25:24 dist-test-slave-65mx krb5kdc[12643](info): commencing operation
krb5kdc: starting...
W20260809 03:25:25.117105 9403 mini_kdc.cc:121] Time spent starting KDC: real 2.339s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:25 dist-test-slave-65mx krb5kdc[12643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:25.247682 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46173
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:46173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:25.360070 12657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:25.360374 12657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:25.360446 12657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:25.364471 12657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:25.364578 12657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:25.364630 12657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:25.364689 12657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:25.364727 12657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:25.369974 12657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:46173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:25.371057 12657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:25.372129 12657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:25.380522 12662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:25.380517 12665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:25.380514 12663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:25.380919 12657 server_base.cc:1034] running on GCE node
I20260809 03:25:25.381485 12657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:25.382638 12657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:25.383827 12657 hybrid_clock.cc:648] HybridClock initialized: now 1786245925383809 us; error 30 us; skew 500 ppm
Aug 09 03:25:25 dist-test-slave-65mx krb5kdc[12643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245925, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:25.390022 12657 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:25.390951 12657 webserver.cc:460] Webserver started at http://127.9.46.254:43323/ using document root <none> and password file <none>
I20260809 03:25:25.391515 12657 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:25.391606 12657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:25.391851 12657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:25.393996 12657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "a07fd7c7dc714b169d5d0b6bded65d9a"
format_stamp: "Formatted at 2026-08-09 03:25:25 on dist-test-slave-65mx"
server_key: "2333879cb61f95ed82b31ee3edf54d5c"
server_key_iv: "a9fa33ac07e256b3951d9df7b926e321"
server_key_version: "encryptionkey@0"
I20260809 03:25:25.394578 12657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "a07fd7c7dc714b169d5d0b6bded65d9a"
format_stamp: "Formatted at 2026-08-09 03:25:25 on dist-test-slave-65mx"
server_key: "2333879cb61f95ed82b31ee3edf54d5c"
server_key_iv: "a9fa33ac07e256b3951d9df7b926e321"
server_key_version: "encryptionkey@0"
I20260809 03:25:25.398406 12657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 03:25:25.400810 12672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:25.402043 12657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:25:25.402211 12657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "a07fd7c7dc714b169d5d0b6bded65d9a"
format_stamp: "Formatted at 2026-08-09 03:25:25 on dist-test-slave-65mx"
server_key: "2333879cb61f95ed82b31ee3edf54d5c"
server_key_iv: "a9fa33ac07e256b3951d9df7b926e321"
server_key_version: "encryptionkey@0"
I20260809 03:25:25.402328 12657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:25.435804 12657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:25.439013 12657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:25.439239 12657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:25.446635 12657 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:46173
I20260809 03:25:25.446964 12726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:46173 every 8 connection(s)
I20260809 03:25:25.447633 12657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:25.450416 12727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:25.455067 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 12657
I20260809 03:25:25.455190 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:25.455487 9403 external_mini_cluster.cc:1428] Setting key 0919adb69c35bfc7a89934c9c7df6776
I20260809 03:25:25.456780 12727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a: Bootstrap starting.
I20260809 03:25:25.459199 12727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:25.460064 12727 log.cc:824] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:25.462044 12727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a: No bootstrap required, opened a new log
Aug 09 03:25:25 dist-test-slave-65mx krb5kdc[12643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:25.465765 12727 raft_consensus.cc:348] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07fd7c7dc714b169d5d0b6bded65d9a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46173 } }
I20260809 03:25:25.466063 12727 raft_consensus.cc:374] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:25.466187 12727 raft_consensus.cc:729] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07fd7c7dc714b169d5d0b6bded65d9a, State: Initialized, Role: FOLLOWER
I20260809 03:25:25.466701 12727 consensus_queue.cc:260] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07fd7c7dc714b169d5d0b6bded65d9a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46173 } }
I20260809 03:25:25.466869 12727 raft_consensus.cc:388] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:25.466938 12727 raft_consensus.cc:482] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:25.467031 12727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:25.468173 12727 raft_consensus.cc:504] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07fd7c7dc714b169d5d0b6bded65d9a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46173 } }
I20260809 03:25:25.468545 12727 leader_election.cc:302] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a07fd7c7dc714b169d5d0b6bded65d9a; no voters:
I20260809 03:25:25.468894 12727 leader_election.cc:288] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:25.469008 12732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:25.469305 12732 raft_consensus.cc:686] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [term 1 LEADER]: Becoming Leader. State: Replica: a07fd7c7dc714b169d5d0b6bded65d9a, State: Running, Role: LEADER
I20260809 03:25:25.469743 12732 consensus_queue.cc:237] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07fd7c7dc714b169d5d0b6bded65d9a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46173 } }
I20260809 03:25:25.470168 12727 sys_catalog.cc:564] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:25.470934 12732 sys_catalog.cc:455] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a07fd7c7dc714b169d5d0b6bded65d9a. Latest consensus state: current_term: 1 leader_uuid: "a07fd7c7dc714b169d5d0b6bded65d9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07fd7c7dc714b169d5d0b6bded65d9a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46173 } } }
I20260809 03:25:25.471299 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:25.457025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.9.46.254:46173)
0809 03:25:25.457426 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:25.457442 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:25.457477 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:25:25.458106 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:25.458122 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:25.458445 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:25.458720 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:25.459733 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:25.460802 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:25.461654 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:25.462159 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:25.462333 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:25.464900 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:25.464905 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:25.464917 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:25.464944 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:25.467461 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:25.467886 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:25.467892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:25.467898 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:25.468059 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:25.468354 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:25.468357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:25.468359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:25.468947 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:25.469187 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:25.469973 (+ 786us) server_negotiation.cc:299] Negotiation successful
0809 03:25:25.470297 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":129,"threads_started":1}
I20260809 03:25:25.471540 12734 sys_catalog.cc:455] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a07fd7c7dc714b169d5d0b6bded65d9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a07fd7c7dc714b169d5d0b6bded65d9a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46173 } } }
I20260809 03:25:25.471909 12734 sys_catalog.cc:458] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:25.471661 12732 sys_catalog.cc:458] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:25.472920 12747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:25.475347 12747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:25.480326 12747 catalog_manager.cc:1347] Generated new cluster ID: c4ec0888bc8f4e7285a1ca14d3b65b64
I20260809 03:25:25.480392 12747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:25.489733 12747 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:25.490671 12747 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:25.506418 12747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a07fd7c7dc714b169d5d0b6bded65d9a: Generated new TSK 0
I20260809 03:25:25.507326 12747 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:25:25.547581 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46173
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:25.661366 12754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:25.661645 12754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:25.661689 12754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:25.665863 12754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:25.665927 12754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:25.666011 12754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:25.671530 12754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:25.672725 12754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:25.673787 12754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:25.683506 12763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:25.683511 12761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:25.683573 12760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:25.683895 12754 server_base.cc:1034] running on GCE node
I20260809 03:25:25.684332 12754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:25.685088 12754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:25.686286 12754 hybrid_clock.cc:648] HybridClock initialized: now 1786245925686273 us; error 25 us; skew 500 ppm
Aug 09 03:25:25 dist-test-slave-65mx krb5kdc[12643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245925, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:25.689082 12754 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:25:25.689937 12754 webserver.cc:460] Webserver started at http://127.9.46.193:33475/ using document root <none> and password file <none>
I20260809 03:25:25.690505 12754 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:25.690615 12754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:25.690840 12754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:25.693025 12754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "d53bd54441fb450f95b44bc04b993a63"
format_stamp: "Formatted at 2026-08-09 03:25:25 on dist-test-slave-65mx"
server_key: "3dd40d842e61a00d427560677c1fcdda"
server_key_iv: "d2c21bfe705ce6b5cea9477bccad411a"
server_key_version: "encryptionkey@0"
I20260809 03:25:25.693540 12754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "d53bd54441fb450f95b44bc04b993a63"
format_stamp: "Formatted at 2026-08-09 03:25:25 on dist-test-slave-65mx"
server_key: "3dd40d842e61a00d427560677c1fcdda"
server_key_iv: "d2c21bfe705ce6b5cea9477bccad411a"
server_key_version: "encryptionkey@0"
I20260809 03:25:25.697149 12754 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:25:25.699395 12770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:25.700675 12754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:25.700789 12754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "d53bd54441fb450f95b44bc04b993a63"
format_stamp: "Formatted at 2026-08-09 03:25:25 on dist-test-slave-65mx"
server_key: "3dd40d842e61a00d427560677c1fcdda"
server_key_iv: "d2c21bfe705ce6b5cea9477bccad411a"
server_key_version: "encryptionkey@0"
I20260809 03:25:25.700899 12754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:25.709630 12754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:25.712266 12754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:25.712474 12754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:25.713079 12754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:25.713887 12754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:25.713934 12754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:25.714006 12754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:25.714038 12754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:25.723510 12754 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:33433
I20260809 03:25:25.723541 12890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:33433 every 8 connection(s)
I20260809 03:25:25.724402 12754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:25.724737 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 12754
I20260809 03:25:25.724877 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:25.725198 9403 external_mini_cluster.cc:1428] Setting key 17fe27ae044b8a27685f4a4d5635e7f0
Aug 09 03:25:25 dist-test-slave-65mx krb5kdc[12643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245925, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
I20260809 03:25:25.740123 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:25.726320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:34293 (local address 127.9.46.254:46173)
0809 03:25:25.726456 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:25.726459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:25.727497 (+ 1038us) server_negotiation.cc:407] Connection header received
0809 03:25:25.729454 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:25.729460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:25.729523 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:25.729640 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:25.730813 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:25.731744 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:25.733057 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:25.733505 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:25.733658 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:25.736251 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:25.736255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:25.736257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:25.736290 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:25.737677 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:25.738312 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:25.738315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:25.738317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:25.738403 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:25.738823 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:25.738827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:25.738830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:25.739037 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:25.739169 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:25.739702 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 03:25:25.739858 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:25:25.740926 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:25.726782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46173 (local address 127.9.46.193:34293)
0809 03:25:25.727312 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:25.727373 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:25.729225 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:25.729793 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:25.729800 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:25.730182 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:25.730656 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:25.730670 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:25.731911 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:25.731916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:25.732931 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:25.732941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:25.733678 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:25.733680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:25.733826 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:25.734475 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:25.734492 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:25.736059 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:25.737803 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:25.737810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:25.737826 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:25.738200 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:25.738500 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:25.738504 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:25.738506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:25.738711 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:25.739180 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:25.739186 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:25.739428 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 03:25:25.739619 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 03:25:25.739897 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":113,"threads_started":1}
I20260809 03:25:25.742275 12891 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46173
I20260809 03:25:25.742519 12891 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:25.743031 12891 heartbeater.cc:511] Master 127.9.46.254:46173 requested a full tablet report, sending...
I20260809 03:25:25.744621 12689 ts_manager.cc:194] Registered new tserver with Master: d53bd54441fb450f95b44bc04b993a63 (127.9.46.193:33433)
I20260809 03:25:25.745870 12689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:34293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:25:25.756510 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46173
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:25.868315 12896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:25.868651 12896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:25.868701 12896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:25.872870 12896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:25.872926 12896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:25.872983 12896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:25.878211 12896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:25.879244 12896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:25.880379 12896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:25.890045 12905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:25.890079 12902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:25.890132 12896 server_base.cc:1034] running on GCE node
W20260809 03:25:25.890046 12903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:25.890743 12896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:25.891412 12896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:25.892632 12896 hybrid_clock.cc:648] HybridClock initialized: now 1786245925892618 us; error 29 us; skew 500 ppm
Aug 09 03:25:25 dist-test-slave-65mx krb5kdc[12643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245925, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:25.895267 12896 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:25:25.896086 12896 webserver.cc:460] Webserver started at http://127.9.46.194:45993/ using document root <none> and password file <none>
I20260809 03:25:25.896692 12896 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:25.896785 12896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:25.897027 12896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:25.898989 12896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "c4b7d7e1be1649e4a52a4d6ab61e2815"
format_stamp: "Formatted at 2026-08-09 03:25:25 on dist-test-slave-65mx"
server_key: "05012f8f5a98ce4a08b4e8b9dca99f35"
server_key_iv: "7c4fdebed467a1d0e1795774e92543e2"
server_key_version: "encryptionkey@0"
I20260809 03:25:25.899533 12896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "c4b7d7e1be1649e4a52a4d6ab61e2815"
format_stamp: "Formatted at 2026-08-09 03:25:25 on dist-test-slave-65mx"
server_key: "05012f8f5a98ce4a08b4e8b9dca99f35"
server_key_iv: "7c4fdebed467a1d0e1795774e92543e2"
server_key_version: "encryptionkey@0"
I20260809 03:25:25.903514 12896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:25:25.906582 12912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:25.907802 12896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:25.907923 12896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "c4b7d7e1be1649e4a52a4d6ab61e2815"
format_stamp: "Formatted at 2026-08-09 03:25:25 on dist-test-slave-65mx"
server_key: "05012f8f5a98ce4a08b4e8b9dca99f35"
server_key_iv: "7c4fdebed467a1d0e1795774e92543e2"
server_key_version: "encryptionkey@0"
I20260809 03:25:25.908074 12896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:25.934149 12896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:25.937248 12896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:25.937455 12896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:25.938073 12896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:25.938907 12896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:25.938954 12896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:25.939031 12896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:25.939070 12896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:25.948757 12896 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:41915
I20260809 03:25:25.948786 13032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:41915 every 8 connection(s)
I20260809 03:25:25.949652 12896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:25.953647 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 12896
I20260809 03:25:25.953749 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:25.954041 9403 external_mini_cluster.cc:1428] Setting key 2f2b05a570b2e460229ec293f683b51f
Aug 09 03:25:25 dist-test-slave-65mx krb5kdc[12643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245925, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:25:25.964164 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:25.951457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:48355 (local address 127.9.46.254:46173)
0809 03:25:25.951610 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:25.951614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:25.952427 (+ 813us) server_negotiation.cc:407] Connection header received
0809 03:25:25.953912 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:25.953915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:25.953976 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:25.954094 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:25.955573 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:25.956351 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:25.957596 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:25.957824 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:25.957939 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:25.960470 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:25.960474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:25.960476 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:25.960508 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:25.962091 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:25.962625 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:25.962628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:25.962629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:25.962703 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:25.963056 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:25.963061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:25.963063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:25.963260 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:25.963373 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:25.963868 (+ 495us) server_negotiation.cc:299] Negotiation successful
0809 03:25:25.963998 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:25:25.965135 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:25.951833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46173 (local address 127.9.46.194:48355)
0809 03:25:25.952275 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:25.952317 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:25.953626 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:25.954258 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:25.954266 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:25.954774 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:25.955410 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:25.955431 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:25.956480 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:25.956483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:25.957438 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:25.957448 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:25.957978 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:25.957982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:25.958158 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:25.958781 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:25.958797 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:25.960309 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:25.962220 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:25.962226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:25.962241 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:25.962501 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:25.962805 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:25.962808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:25.962810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:25.962956 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:25.963371 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:25.963379 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:25.963626 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 03:25:25.963815 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 03:25:25.964154 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":112,"threads_started":1}
I20260809 03:25:25.966277 13033 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46173
I20260809 03:25:25.966529 13033 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
I20260809 03:25:25.967168 13033 heartbeater.cc:511] Master 127.9.46.254:46173 requested a full tablet report, sending...
I20260809 03:25:25.968278 12689 ts_manager.cc:194] Registered new tserver with Master: c4b7d7e1be1649e4a52a4d6ab61e2815 (127.9.46.194:41915)
I20260809 03:25:25.968914 12689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:48355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:25:25.983801 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46173
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:26.097643 13038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:26.097931 13038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:26.098012 13038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:26.101994 13038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:26.102088 13038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:26.102171 13038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:26.107638 13038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:26.108686 13038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:26.109676 13038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:26.119052 13046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:26.119024 13044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:26.119082 13043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:26.119421 13038 server_base.cc:1034] running on GCE node
I20260809 03:25:26.119807 13038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:26.120430 13038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:26.121620 13038 hybrid_clock.cc:648] HybridClock initialized: now 1786245926121587 us; error 44 us; skew 500 ppm
Aug 09 03:25:26 dist-test-slave-65mx krb5kdc[12643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245926, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:26.125494 13038 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:25:26.126379 13038 webserver.cc:460] Webserver started at http://127.9.46.195:35011/ using document root <none> and password file <none>
I20260809 03:25:26.126922 13038 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:26.127012 13038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:26.127264 13038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:26.129330 13038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "4f009beaf71640d9bf61bb30d9a2ba04"
format_stamp: "Formatted at 2026-08-09 03:25:26 on dist-test-slave-65mx"
server_key: "c100e054a5cb7345c415e9da940b2734"
server_key_iv: "1c0dec731d7d8539e6a0162dc7925a88"
server_key_version: "encryptionkey@0"
I20260809 03:25:26.129894 13038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "4f009beaf71640d9bf61bb30d9a2ba04"
format_stamp: "Formatted at 2026-08-09 03:25:26 on dist-test-slave-65mx"
server_key: "c100e054a5cb7345c415e9da940b2734"
server_key_iv: "1c0dec731d7d8539e6a0162dc7925a88"
server_key_version: "encryptionkey@0"
I20260809 03:25:26.133524 13038 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:26.135800 13054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:26.137074 13038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:26.137226 13038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "4f009beaf71640d9bf61bb30d9a2ba04"
format_stamp: "Formatted at 2026-08-09 03:25:26 on dist-test-slave-65mx"
server_key: "c100e054a5cb7345c415e9da940b2734"
server_key_iv: "1c0dec731d7d8539e6a0162dc7925a88"
server_key_version: "encryptionkey@0"
I20260809 03:25:26.137337 13038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:26.150131 13038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:26.153014 13038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:26.153211 13038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:26.153767 13038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:26.154570 13038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:26.154641 13038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:26.154712 13038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:26.154764 13038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:26.163955 13038 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:44903
I20260809 03:25:26.163991 13173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:44903 every 8 connection(s)
I20260809 03:25:26.164934 13038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:26.170480 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 13038
I20260809 03:25:26.170639 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:26.170915 9403 external_mini_cluster.cc:1428] Setting key eb2aca7e8fe1596fee3fc3f0be210d1e
Aug 09 03:25:26 dist-test-slave-65mx krb5kdc[12643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245926, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:26.179193 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:26.166724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:59977 (local address 127.9.46.254:46173)
0809 03:25:26.166859 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:26.166862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:26.167621 (+ 759us) server_negotiation.cc:407] Connection header received
0809 03:25:26.169054 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:26.169058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:26.169146 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:26.169242 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:26.170605 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:26.171318 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:26.173015 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:26.173314 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:26.173400 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:26.175859 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:26.175861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:26.175863 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:26.175885 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:26.177312 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:26.177765 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:26.177768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:26.177769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:26.177860 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:26.178210 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:26.178212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:26.178213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:26.178434 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:26.178513 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:26.178964 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 03:25:26.179072 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:25:26.179966 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:26.167032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46173 (local address 127.9.46.195:59977)
0809 03:25:26.167460 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:26.167501 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:26.168850 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:26.169406 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:26.169412 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:26.169794 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:26.170407 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:26.170429 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:26.171475 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:26.171478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:26.172874 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:26.172884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:26.173454 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:26.173457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:26.173591 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:26.174250 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:26.174267 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:26.175714 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:26.177413 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:26.177418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:26.177428 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:26.177682 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:26.177951 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:26.177954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:26.177955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:26.178125 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:26.178523 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:26.178528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:26.178766 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 03:25:26.178916 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 03:25:26.179158 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":103,"threads_started":1}
I20260809 03:25:26.181133 13174 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46173
I20260809 03:25:26.181361 13174 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:26.181840 13174 heartbeater.cc:511] Master 127.9.46.254:46173 requested a full tablet report, sending...
I20260809 03:25:26.182832 12689 ts_manager.cc:194] Registered new tserver with Master: 4f009beaf71640d9bf61bb30d9a2ba04 (127.9.46.195:44903)
I20260809 03:25:26.183456 12689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:59977
I20260809 03:25:26.184937 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:25:26 dist-test-slave-65mx krb5kdc[12643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245926, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 03:25:26 dist-test-slave-65mx krb5kdc[12643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245926, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:26.211871 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:26.202903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43520 (local address 127.9.46.254:46173)
0809 03:25:26.203081 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:26.203085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:26.203241 (+ 156us) server_negotiation.cc:407] Connection header received
0809 03:25:26.203401 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:26.203404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:26.203455 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:26.203545 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:26.204267 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:26.204930 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:26.205911 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:26.206161 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:26.206271 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:26.208722 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:26.208728 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:26.208735 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:26.208760 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:26.210040 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:26.210484 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:26.210487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:26.210489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:26.210575 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:26.210855 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:26.210859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:26.210861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:26.211178 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:26.211317 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:26.211580 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 03:25:26.211698 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 03:25:26.212262 12689 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43520
I20260809 03:25:26.213121 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 12754
I20260809 03:25:26.220418 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 12896
I20260809 03:25:26.227962 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 13038
I20260809 03:25:26.234727 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 12657
2026-08-09T03:25:26Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3479 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:26 dist-test-slave-65mx krb5kdc[13186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:26 dist-test-slave-65mx krb5kdc[13186](info): set up 2 sockets
Aug 09 03:25:26 dist-test-slave-65mx krb5kdc[13186](info): commencing operation
krb5kdc: starting...
W20260809 03:25:27.336699 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:27 dist-test-slave-65mx krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:27.467921 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39477
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:40917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:39477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:27.581040 13199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:27.581305 13199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:27.581346 13199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:27.585368 13199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:27.585425 13199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:27.585445 13199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:27.585461 13199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:27.585477 13199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:27.590689 13199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:40917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:39477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:27.591732 13199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:27.592829 13199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:27.601298 13205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:27.601321 13207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:27.601312 13204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:27.601858 13199 server_base.cc:1034] running on GCE node
I20260809 03:25:27.602401 13199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:27.603472 13199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:27.604641 13199 hybrid_clock.cc:648] HybridClock initialized: now 1786245927604606 us; error 45 us; skew 500 ppm
Aug 09 03:25:27 dist-test-slave-65mx krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245927, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:27.607532 13199 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:27.608448 13199 webserver.cc:460] Webserver started at http://127.9.46.254:42723/ using document root <none> and password file <none>
I20260809 03:25:27.609076 13199 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:27.609143 13199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:27.609315 13199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:27.611418 13199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "24c1af4bdfda4cea8020fc1490180d7c"
format_stamp: "Formatted at 2026-08-09 03:25:27 on dist-test-slave-65mx"
server_key: "884d7f0cf7f0df99dd2d1be2d49cf0ff"
server_key_iv: "8e4511c74f43922ec9d392f70ee47698"
server_key_version: "encryptionkey@0"
I20260809 03:25:27.611943 13199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "24c1af4bdfda4cea8020fc1490180d7c"
format_stamp: "Formatted at 2026-08-09 03:25:27 on dist-test-slave-65mx"
server_key: "884d7f0cf7f0df99dd2d1be2d49cf0ff"
server_key_iv: "8e4511c74f43922ec9d392f70ee47698"
server_key_version: "encryptionkey@0"
I20260809 03:25:27.615778 13199 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:25:27.618281 13214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:27.619539 13199 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 03:25:27.619644 13199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "24c1af4bdfda4cea8020fc1490180d7c"
format_stamp: "Formatted at 2026-08-09 03:25:27 on dist-test-slave-65mx"
server_key: "884d7f0cf7f0df99dd2d1be2d49cf0ff"
server_key_iv: "8e4511c74f43922ec9d392f70ee47698"
server_key_version: "encryptionkey@0"
I20260809 03:25:27.619730 13199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:27.638868 13199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:27.641986 13199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:27.642160 13199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:27.650171 13199 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:39477
I20260809 03:25:27.650200 13270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:39477 every 8 connection(s)
I20260809 03:25:27.651247 13199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:27.654276 13271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:27.655303 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 13199
I20260809 03:25:27.655459 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:27.655762 9403 external_mini_cluster.cc:1428] Setting key a2675526dddaf5b3f70731c8feb6dad5
I20260809 03:25:27.661427 13271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c: Bootstrap starting.
I20260809 03:25:27.664212 13271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:27.665104 13271 log.cc:824] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:25:27 dist-test-slave-65mx krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:27.667292 13271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c: No bootstrap required, opened a new log
I20260809 03:25:27.671778 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:27.657341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35870 (local address 127.9.46.254:39477)
0809 03:25:27.657807 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:27.657828 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:27.657873 (+ 45us) server_negotiation.cc:407] Connection header received
0809 03:25:27.658759 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:27.658783 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:27.659214 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:27.659586 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:27.660288 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:27.661780 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:27.662633 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:27.663332 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:27.663503 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:27.666340 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:27.666345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:27.666358 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:27.666383 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:27.668451 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:27.668948 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:27.668953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:27.668960 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:27.669116 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:27.669405 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:27.669409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:27.669411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:27.670017 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:27.670214 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:27.670474 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 03:25:27.670785 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":123,"threads_started":1}
I20260809 03:25:27.672125 13271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c1af4bdfda4cea8020fc1490180d7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39477 } }
I20260809 03:25:27.672370 13271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:27.672428 13271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c1af4bdfda4cea8020fc1490180d7c, State: Initialized, Role: FOLLOWER
I20260809 03:25:27.672876 13271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c1af4bdfda4cea8020fc1490180d7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39477 } }
I20260809 03:25:27.673034 13271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:27.673121 13271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:27.673250 13271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:27.674213 13271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c1af4bdfda4cea8020fc1490180d7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39477 } }
I20260809 03:25:27.674557 13271 leader_election.cc:302] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24c1af4bdfda4cea8020fc1490180d7c; no voters:
I20260809 03:25:27.674885 13271 leader_election.cc:288] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:27.675127 13276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:27.675390 13276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [term 1 LEADER]: Becoming Leader. State: Replica: 24c1af4bdfda4cea8020fc1490180d7c, State: Running, Role: LEADER
I20260809 03:25:27.675715 13276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c1af4bdfda4cea8020fc1490180d7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39477 } }
I20260809 03:25:27.676204 13271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:27.677559 13278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24c1af4bdfda4cea8020fc1490180d7c. Latest consensus state: current_term: 1 leader_uuid: "24c1af4bdfda4cea8020fc1490180d7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c1af4bdfda4cea8020fc1490180d7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39477 } } }
I20260809 03:25:27.677659 13277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24c1af4bdfda4cea8020fc1490180d7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c1af4bdfda4cea8020fc1490180d7c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39477 } } }
I20260809 03:25:27.677721 13278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:27.677788 13277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:27.678189 13285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:27.681399 13285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:27.692521 13285 catalog_manager.cc:1347] Generated new cluster ID: ee8ddf43a5274c589c30e1999be90ec8
I20260809 03:25:27.692660 13285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:27.708099 13285 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:27.709406 13285 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:27.721632 13285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24c1af4bdfda4cea8020fc1490180d7c: Generated new TSK 0
I20260809 03:25:27.722412 13285 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:25:27.760327 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39477
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:40917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:27.876888 13297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:27.877187 13297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:27.877233 13297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:27.881498 13297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:27.881561 13297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:27.881619 13297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:27.886994 13297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:40917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:27.888021 13297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:27.889098 13297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:27.898605 13303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:27.898622 13305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:27.898568 13297 server_base.cc:1034] running on GCE node
W20260809 03:25:27.898591 13302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:27.899222 13297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:27.899883 13297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:27.901072 13297 hybrid_clock.cc:648] HybridClock initialized: now 1786245927901052 us; error 33 us; skew 500 ppm
Aug 09 03:25:27 dist-test-slave-65mx krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245927, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:27.903826 13297 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:25:27.904731 13297 webserver.cc:460] Webserver started at http://127.9.46.193:44613/ using document root <none> and password file <none>
I20260809 03:25:27.905282 13297 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:27.905375 13297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:27.905604 13297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:27.907591 13297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "c7af91fcd991486499583891b86064f1"
format_stamp: "Formatted at 2026-08-09 03:25:27 on dist-test-slave-65mx"
server_key: "a3361aff6ebd958364b2c68d4d38062f"
server_key_iv: "45d777b1fd4cc52deeb0489cc8a45e93"
server_key_version: "encryptionkey@0"
I20260809 03:25:27.908166 13297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "c7af91fcd991486499583891b86064f1"
format_stamp: "Formatted at 2026-08-09 03:25:27 on dist-test-slave-65mx"
server_key: "a3361aff6ebd958364b2c68d4d38062f"
server_key_iv: "45d777b1fd4cc52deeb0489cc8a45e93"
server_key_version: "encryptionkey@0"
I20260809 03:25:27.912624 13297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 03:25:27.915375 13313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:27.916679 13297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:27.916798 13297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "c7af91fcd991486499583891b86064f1"
format_stamp: "Formatted at 2026-08-09 03:25:27 on dist-test-slave-65mx"
server_key: "a3361aff6ebd958364b2c68d4d38062f"
server_key_iv: "45d777b1fd4cc52deeb0489cc8a45e93"
server_key_version: "encryptionkey@0"
I20260809 03:25:27.916911 13297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:27.940119 13297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:27.943717 13297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:27.943884 13297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:27.944590 13297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:27.945482 13297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:27.945559 13297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:27.945634 13297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:27.945688 13297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:27.955430 13297 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:34287
I20260809 03:25:27.955453 13433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:34287 every 8 connection(s)
I20260809 03:25:27.956338 13297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:27.957957 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 13297
I20260809 03:25:27.958097 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:27.958449 9403 external_mini_cluster.cc:1428] Setting key 891c30d54497bfa94e98eca767122c05
Aug 09 03:25:27 dist-test-slave-65mx krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245927, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:25:27.970995 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:27.958414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:35205 (local address 127.9.46.254:39477)
0809 03:25:27.958559 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:27.958563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:27.959530 (+ 967us) server_negotiation.cc:407] Connection header received
0809 03:25:27.961087 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:27.961090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:27.961153 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:27.961234 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:27.962389 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:27.962922 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:27.964087 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:27.964494 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:27.964641 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:27.967192 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:27.967197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:27.967202 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:27.967238 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:27.968953 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:27.969469 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:27.969473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:27.969475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:27.969576 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:27.969923 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:27.969926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:27.969928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:27.970166 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:27.970283 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:27.970770 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 03:25:27.970874 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:25:27.971800 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:27.958877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:39477 (local address 127.9.46.193:35205)
0809 03:25:27.959388 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:27.959431 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:27.960876 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:27.961403 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:27.961409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:27.961763 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:27.962235 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:27.962248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:27.963052 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:27.963056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:27.963947 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:27.963957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:27.964676 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:27.964680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:27.964831 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:27.965452 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:27.965468 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:27.966995 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:27.969078 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:27.969083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:27.969093 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:27.969354 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:27.969683 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:27.969686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:27.969687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:27.969816 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:27.970300 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:27.970306 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:27.970546 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 03:25:27.970705 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 03:25:27.970961 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":137,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
I20260809 03:25:27.973227 13434 heartbeater.cc:347] Connected to a master server at 127.9.46.254:39477
I20260809 03:25:27.973505 13434 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:27.973990 13434 heartbeater.cc:511] Master 127.9.46.254:39477 requested a full tablet report, sending...
I20260809 03:25:27.975528 13232 ts_manager.cc:194] Registered new tserver with Master: c7af91fcd991486499583891b86064f1 (127.9.46.193:34287)
I20260809 03:25:27.977198 13232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:35205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:25:27.991410 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39477
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:40917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:28.106032 13439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:28.106333 13439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:28.106446 13439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:28.110561 13439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:28.110646 13439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:28.110733 13439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:28.116040 13439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:40917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:28.117167 13439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:28.118242 13439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:28.127878 13449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:28.127930 13447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:28.127930 13446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:28.128227 13439 server_base.cc:1034] running on GCE node
I20260809 03:25:28.128691 13439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:28.129308 13439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:28.130501 13439 hybrid_clock.cc:648] HybridClock initialized: now 1786245928130464 us; error 48 us; skew 500 ppm
Aug 09 03:25:28 dist-test-slave-65mx krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:28.133281 13439 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:25:28.134045 13439 webserver.cc:460] Webserver started at http://127.9.46.194:40773/ using document root <none> and password file <none>
I20260809 03:25:28.134548 13439 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:28.134609 13439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:28.134769 13439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:28.136765 13439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "56056788b0994d42898ab34915210f64"
format_stamp: "Formatted at 2026-08-09 03:25:28 on dist-test-slave-65mx"
server_key: "502acb83073f9525348f2dab86bf325e"
server_key_iv: "cfd2735f670fa9fbb47947b8425a3d6c"
server_key_version: "encryptionkey@0"
I20260809 03:25:28.137358 13439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "56056788b0994d42898ab34915210f64"
format_stamp: "Formatted at 2026-08-09 03:25:28 on dist-test-slave-65mx"
server_key: "502acb83073f9525348f2dab86bf325e"
server_key_iv: "cfd2735f670fa9fbb47947b8425a3d6c"
server_key_version: "encryptionkey@0"
I20260809 03:25:28.141038 13439 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:28.143376 13457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:28.144601 13439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:28.144704 13439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "56056788b0994d42898ab34915210f64"
format_stamp: "Formatted at 2026-08-09 03:25:28 on dist-test-slave-65mx"
server_key: "502acb83073f9525348f2dab86bf325e"
server_key_iv: "cfd2735f670fa9fbb47947b8425a3d6c"
server_key_version: "encryptionkey@0"
I20260809 03:25:28.144785 13439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:28.171496 13439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:28.175100 13439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:28.175307 13439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:28.175918 13439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:28.176874 13439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:28.176994 13439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:28.177094 13439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:28.177171 13439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 03:25:28.186453 13439 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:32789
I20260809 03:25:28.186483 13575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:32789 every 8 connection(s)
I20260809 03:25:28.187330 13439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:28.188097 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 13439
I20260809 03:25:28.188236 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:28.188647 9403 external_mini_cluster.cc:1428] Setting key 7a00e1a92d15bf0f1ea50781ac951874
Aug 09 03:25:28 dist-test-slave-65mx krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
I20260809 03:25:28.202627 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:28.189202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:56487 (local address 127.9.46.254:39477)
0809 03:25:28.189353 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:28.189358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:28.190363 (+ 1005us) server_negotiation.cc:407] Connection header received
0809 03:25:28.191743 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:28.191746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:28.191801 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:28.191881 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:28.193091 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.193941 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.195206 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.195631 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.195765 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:28.198616 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:28.198623 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:28.198629 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:28.198667 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:28.200283 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:28.200920 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:28.200923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:28.200925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:28.201007 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:28.201340 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:28.201342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:28.201343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:28.201609 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:28.201699 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:28.202311 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 03:25:28.202429 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:25:28.203518 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:28.189627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:39477 (local address 127.9.46.194:56487)
0809 03:25:28.190227 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:28.190271 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:28.191540 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:28.192043 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:28.192050 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:28.192405 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:28.192945 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:28.192959 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.194085 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.194088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:28.195070 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:28.195081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.195800 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.195802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:28.195943 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:28.196648 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:28.196664 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:28.198419 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:28.200407 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:28.200413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:28.200478 (+ 65us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:28.200822 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:28.201102 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:28.201105 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:28.201106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:28.201251 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:28.201699 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:28.201705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:28.201983 (+ 278us) client_negotiation.cc:769] Sending connection context
0809 03:25:28.202222 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 03:25:28.202596 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":112,"threads_started":1}
I20260809 03:25:28.204854 13576 heartbeater.cc:347] Connected to a master server at 127.9.46.254:39477
I20260809 03:25:28.205150 13576 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:28.205790 13576 heartbeater.cc:511] Master 127.9.46.254:39477 requested a full tablet report, sending...
I20260809 03:25:28.207015 13232 ts_manager.cc:194] Registered new tserver with Master: 56056788b0994d42898ab34915210f64 (127.9.46.194:32789)
I20260809 03:25:28.207948 13232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:56487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:25:28.219947 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39477
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:40917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:28.332091 13582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:28.332399 13582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:28.332489 13582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:28.336532 13582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:28.336706 13582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:28.336807 13582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:28.342276 13582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:40917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:28.343395 13582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:28.344467 13582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:28.354583 13588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:28.354535 13587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:28.354538 13590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:28.354820 13582 server_base.cc:1034] running on GCE node
I20260809 03:25:28.355350 13582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:28.356046 13582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:28.357240 13582 hybrid_clock.cc:648] HybridClock initialized: now 1786245928357225 us; error 29 us; skew 500 ppm
Aug 09 03:25:28 dist-test-slave-65mx krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:28.359967 13582 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:25:28.360972 13582 webserver.cc:460] Webserver started at http://127.9.46.195:34219/ using document root <none> and password file <none>
I20260809 03:25:28.361536 13582 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:28.361646 13582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:28.361887 13582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:28.363974 13582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "6aff4e2d3d874761923e2c47f052fca6"
format_stamp: "Formatted at 2026-08-09 03:25:28 on dist-test-slave-65mx"
server_key: "06c886ba8c84f33247d99987abfba677"
server_key_iv: "17bde2a1f2694545cadc503494e1c159"
server_key_version: "encryptionkey@0"
I20260809 03:25:28.364639 13582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "6aff4e2d3d874761923e2c47f052fca6"
format_stamp: "Formatted at 2026-08-09 03:25:28 on dist-test-slave-65mx"
server_key: "06c886ba8c84f33247d99987abfba677"
server_key_iv: "17bde2a1f2694545cadc503494e1c159"
server_key_version: "encryptionkey@0"
I20260809 03:25:28.368468 13582 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:25:28.370980 13597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:28.372246 13582 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:28.372388 13582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "6aff4e2d3d874761923e2c47f052fca6"
format_stamp: "Formatted at 2026-08-09 03:25:28 on dist-test-slave-65mx"
server_key: "06c886ba8c84f33247d99987abfba677"
server_key_iv: "17bde2a1f2694545cadc503494e1c159"
server_key_version: "encryptionkey@0"
I20260809 03:25:28.372509 13582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:28.388525 13582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:28.391435 13582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:28.391657 13582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:28.392277 13582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:28.393169 13582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:28.393242 13582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:28.393316 13582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:28.393370 13582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:28.402643 13582 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:38133
I20260809 03:25:28.402673 13717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:38133 every 8 connection(s)
I20260809 03:25:28.403515 13582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:28.406597 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 13582
I20260809 03:25:28.406746 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:28.407080 9403 external_mini_cluster.cc:1428] Setting key 2ce2ac90a6aed9186df3b3ad81d18c5d
Aug 09 03:25:28 dist-test-slave-65mx krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:28.417744 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:28.405361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:57177 (local address 127.9.46.254:39477)
0809 03:25:28.405519 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:28.405524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:28.406449 (+ 925us) server_negotiation.cc:407] Connection header received
0809 03:25:28.408265 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:28.408267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:28.408315 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:28.408420 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:28.409803 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.410327 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.411444 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.411680 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.411786 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:28.414318 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:28.414321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:28.414323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:28.414348 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:28.415737 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:28.416281 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:28.416284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:28.416286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:28.416383 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:28.416762 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:28.416764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:28.416765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:28.416959 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:28.417039 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:28.417509 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 03:25:28.417610 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:25:28.418534 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:28.405778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:39477 (local address 127.9.46.195:57177)
0809 03:25:28.406231 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:28.406335 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:28.408021 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:28.408653 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:28.408664 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:28.409045 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:28.409676 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:28.409690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.410434 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.410436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:28.411301 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:28.411311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.411852 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.411855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:28.411997 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:28.412620 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:28.412642 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:28.414156 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:28.415844 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:28.415848 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:28.415860 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:28.416134 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:28.416476 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:28.416478 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:28.416479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:28.416676 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:28.417051 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:28.417057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:28.417295 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 03:25:28.417443 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 03:25:28.417690 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":117,"threads_started":1}
I20260809 03:25:28.419685 13718 heartbeater.cc:347] Connected to a master server at 127.9.46.254:39477
I20260809 03:25:28.419968 13718 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:28.420619 13718 heartbeater.cc:511] Master 127.9.46.254:39477 requested a full tablet report, sending...
I20260809 03:25:28.421713 13232 ts_manager.cc:194] Registered new tserver with Master: 6aff4e2d3d874761923e2c47f052fca6 (127.9.46.195:38133)
I20260809 03:25:28.422302 13232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:57177
I20260809 03:25:28.433434 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:25:28 dist-test-slave-65mx krb5kdc[13186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:25:28 dist-test-slave-65mx krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:25:28.461112 13725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:28.451005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.9.46.193:34287)
0809 03:25:28.451364 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:28.451380 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:28.451405 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:25:28.451456 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:28.451461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:28.451661 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:28.451784 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:28.452593 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.453384 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.454340 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.454710 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.454793 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:28.457607 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:28.457618 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:28.457649 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:28.457673 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:28.459225 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:28.459683 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:28.459688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:28.459689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:28.459788 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:28.460067 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:28.460070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:28.460071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:28.460302 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:28.460453 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:28.460734 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 03:25:28.460902 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":150,"threads_started":1}
Aug 09 03:25:28 dist-test-slave-65mx krb5kdc[13186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:28.472069 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:28.464035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.9.46.254:39477)
0809 03:25:28.464197 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:28.464204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:28.464220 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:25:28.464374 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:28.464378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:28.464424 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:28.464520 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:28.465165 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.465686 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.466457 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:28.466696 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:28.466809 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:28.469105 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:28.469109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:28.469111 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:28.469147 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:28.470368 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:28.470794 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:28.470797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:28.470799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:28.470890 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:28.471168 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:28.471171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:28.471173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:28.471398 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:28.471551 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:28.471778 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 03:25:28.471899 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 03:25:28.472507 13232 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35874
I20260809 03:25:28.473157 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 13297
I20260809 03:25:28.480439 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 13439
I20260809 03:25:28.488020 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 13582
I20260809 03:25:28.496100 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 13199
2026-08-09T03:25:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2261 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:28 dist-test-slave-65mx krb5kdc[13731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:28 dist-test-slave-65mx krb5kdc[13731](info): set up 2 sockets
Aug 09 03:25:28 dist-test-slave-65mx krb5kdc[13731](info): commencing operation
krb5kdc: starting...
W20260809 03:25:29.581387 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:29 dist-test-slave-65mx krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:29.708652 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41961
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:38271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:41961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:29.819964 13744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:29.820256 13744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:29.820349 13744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:29.824400 13744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:29.824496 13744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:29.824535 13744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:29.824587 13744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:29.824621 13744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:29.830044 13744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:41961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:29.831205 13744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:29.832286 13744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:29.840794 13750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:29.840790 13752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:29.840786 13744 server_base.cc:1034] running on GCE node
W20260809 03:25:29.840826 13749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:29.841666 13744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:29.842749 13744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:29.843940 13744 hybrid_clock.cc:648] HybridClock initialized: now 1786245929843920 us; error 31 us; skew 500 ppm
Aug 09 03:25:29 dist-test-slave-65mx krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245929, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:29.846820 13744 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:29.847538 13744 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:29.847604 13744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:29.847774 13744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:29.849804 13744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "b121a3d53b7248e2b8b7cfc7964c9f93"
format_stamp: "Formatted at 2026-08-09 03:25:29 on dist-test-slave-65mx"
server_key: "61a152fa2cb170b7f91b3776324e798c"
server_key_iv: "48d0ed817d3ccc62fdcc36d05e44aa10"
server_key_version: "encryptionkey@0"
I20260809 03:25:29.850364 13744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "b121a3d53b7248e2b8b7cfc7964c9f93"
format_stamp: "Formatted at 2026-08-09 03:25:29 on dist-test-slave-65mx"
server_key: "61a152fa2cb170b7f91b3776324e798c"
server_key_iv: "48d0ed817d3ccc62fdcc36d05e44aa10"
server_key_version: "encryptionkey@0"
I20260809 03:25:29.853901 13744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:25:29.856256 13758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:29.857542 13744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:25:29.857681 13744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "b121a3d53b7248e2b8b7cfc7964c9f93"
format_stamp: "Formatted at 2026-08-09 03:25:29 on dist-test-slave-65mx"
server_key: "61a152fa2cb170b7f91b3776324e798c"
server_key_iv: "48d0ed817d3ccc62fdcc36d05e44aa10"
server_key_version: "encryptionkey@0"
I20260809 03:25:29.857789 13744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:29.875747 13744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:29.878952 13744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:29.879184 13744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:29.886608 13744 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:41961
I20260809 03:25:29.886929 13814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:41961 every 8 connection(s)
I20260809 03:25:29.887352 13744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:29.890311 13815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:29.895259 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 13744
I20260809 03:25:29.895412 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:29.895735 9403 external_mini_cluster.cc:1428] Setting key 4b8b78d0069b5a9dd3311d5c186453a6
I20260809 03:25:29.896587 13815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93: Bootstrap starting.
I20260809 03:25:29.899061 13815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:29.900100 13815 log.cc:824] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:29.902462 13815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93: No bootstrap required, opened a new log
Aug 09 03:25:29 dist-test-slave-65mx krb5kdc[13731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:29.907295 13815 raft_consensus.cc:348] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121a3d53b7248e2b8b7cfc7964c9f93" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41961 } }
I20260809 03:25:29.907675 13815 raft_consensus.cc:374] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:29.907780 13815 raft_consensus.cc:729] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b121a3d53b7248e2b8b7cfc7964c9f93, State: Initialized, Role: FOLLOWER
I20260809 03:25:29.908249 13815 consensus_queue.cc:260] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121a3d53b7248e2b8b7cfc7964c9f93" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41961 } }
I20260809 03:25:29.908412 13815 raft_consensus.cc:388] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:29.908509 13815 raft_consensus.cc:482] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:29.908659 13815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:29.909893 13815 raft_consensus.cc:504] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121a3d53b7248e2b8b7cfc7964c9f93" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41961 } }
I20260809 03:25:29.910260 13815 leader_election.cc:302] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b121a3d53b7248e2b8b7cfc7964c9f93; no voters:
I20260809 03:25:29.910666 13815 leader_election.cc:288] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:29.910827 13820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:29.911041 13820 raft_consensus.cc:686] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [term 1 LEADER]: Becoming Leader. State: Replica: b121a3d53b7248e2b8b7cfc7964c9f93, State: Running, Role: LEADER
I20260809 03:25:29.911350 13820 consensus_queue.cc:237] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121a3d53b7248e2b8b7cfc7964c9f93" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41961 } }
I20260809 03:25:29.911813 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:29.897271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.9.46.254:41961)
0809 03:25:29.897733 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:29.897752 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:29.897790 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:25:29.898492 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:29.898511 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:29.898864 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:29.899246 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:29.900248 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:29.901563 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:29.902832 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:29.903311 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:29.903436 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:29.906454 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:29.906460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:29.906473 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:29.906500 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:29.908532 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:29.909124 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:29.909128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:29.909133 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:29.909263 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:29.909574 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:29.909576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:29.909577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:29.909998 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:29.910191 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:29.910522 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 03:25:29.910865 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":152,"threads_started":1}
I20260809 03:25:29.912050 13815 sys_catalog.cc:564] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:29.912636 13820 sys_catalog.cc:455] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b121a3d53b7248e2b8b7cfc7964c9f93. Latest consensus state: current_term: 1 leader_uuid: "b121a3d53b7248e2b8b7cfc7964c9f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121a3d53b7248e2b8b7cfc7964c9f93" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41961 } } }
I20260809 03:25:29.912755 13820 sys_catalog.cc:458] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:29.913067 13822 sys_catalog.cc:455] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b121a3d53b7248e2b8b7cfc7964c9f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121a3d53b7248e2b8b7cfc7964c9f93" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41961 } } }
I20260809 03:25:29.913177 13822 sys_catalog.cc:458] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:29.913879 13824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:29.917169 13824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:29.933174 13824 catalog_manager.cc:1347] Generated new cluster ID: f16f24d1feec483497232de102019cdc
I20260809 03:25:29.933265 13824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:29.960330 13824 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:29.961704 13824 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:29.970827 13824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93: Generated new TSK 0
I20260809 03:25:29.971683 13824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:25:30.009305 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41961
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:38271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:30.123690 13841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:30.123947 13841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:30.123999 13841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:30.128381 13841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:30.128454 13841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:30.128557 13841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:30.133908 13841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:30.134963 13841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:30.136029 13841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:30.145321 13847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:30.145439 13848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:30.145313 13850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:30.145522 13841 server_base.cc:1034] running on GCE node
I20260809 03:25:30.145984 13841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:30.146616 13841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:30.147804 13841 hybrid_clock.cc:648] HybridClock initialized: now 1786245930147772 us; error 43 us; skew 500 ppm
Aug 09 03:25:30 dist-test-slave-65mx krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:30.150684 13841 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:25:30.151474 13841 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:30.151571 13841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:30.151796 13841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:30.153932 13841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "732fc47fad8b410bb8d844fceffdcc3d"
format_stamp: "Formatted at 2026-08-09 03:25:30 on dist-test-slave-65mx"
server_key: "7812450f04cbe6aaf4fdfc52d465f5d7"
server_key_iv: "599d2f01a9f6d6c604b2f8e53fbd60f9"
server_key_version: "encryptionkey@0"
I20260809 03:25:30.154506 13841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "732fc47fad8b410bb8d844fceffdcc3d"
format_stamp: "Formatted at 2026-08-09 03:25:30 on dist-test-slave-65mx"
server_key: "7812450f04cbe6aaf4fdfc52d465f5d7"
server_key_iv: "599d2f01a9f6d6c604b2f8e53fbd60f9"
server_key_version: "encryptionkey@0"
I20260809 03:25:30.158355 13841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:25:30.160784 13856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:30.162055 13841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:30.162189 13841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "732fc47fad8b410bb8d844fceffdcc3d"
format_stamp: "Formatted at 2026-08-09 03:25:30 on dist-test-slave-65mx"
server_key: "7812450f04cbe6aaf4fdfc52d465f5d7"
server_key_iv: "599d2f01a9f6d6c604b2f8e53fbd60f9"
server_key_version: "encryptionkey@0"
I20260809 03:25:30.162310 13841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:30.181847 13841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:30.184726 13841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:30.184942 13841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:30.185539 13841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:30.186372 13841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:30.186444 13841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:30.186517 13841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:30.186570 13841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:30.195364 13841 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:37769
I20260809 03:25:30.195389 13973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:37769 every 8 connection(s)
I20260809 03:25:30.195945 13841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:30.196537 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 13841
I20260809 03:25:30.196707 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:30.197100 9403 external_mini_cluster.cc:1428] Setting key 52386f252ee1cc80ded7d678fe4fdffd
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
Aug 09 03:25:30 dist-test-slave-65mx krb5kdc[13731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
I20260809 03:25:30.213794 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.197892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:52717 (local address 127.9.46.254:41961)
0809 03:25:30.198015 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:30.198019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:30.200958 (+ 2939us) server_negotiation.cc:407] Connection header received
0809 03:25:30.202169 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:30.202172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:30.202220 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:30.202326 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:30.204054 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.204659 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.206507 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.206745 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.206826 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.209746 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:30.209749 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:30.209751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:30.209777 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:30.211436 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:30.211992 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:30.211996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:30.211998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:30.212133 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:30.212491 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:30.212496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:30.212498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:30.212735 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:30.212847 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:30.213457 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 03:25:30.213602 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:25:30.214664 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.198356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41961 (local address 127.9.46.193:52717)
0809 03:25:30.200794 (+ 2438us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:30.200840 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:30.201955 (+ 1115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:30.202517 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:30.202528 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:30.203066 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:30.203876 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.203901 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.204790 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.204796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.206375 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.206388 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.206920 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.206925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.207151 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.207807 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:30.207830 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:30.209552 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:30.211569 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:30.211576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:30.211592 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:30.211902 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:30.212221 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:30.212224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:30.212226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:30.212387 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:30.212848 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:30.212856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:30.213112 (+ 256us) client_negotiation.cc:769] Sending connection context
0809 03:25:30.213398 (+ 286us) client_negotiation.cc:240] Negotiation successful
0809 03:25:30.213787 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2230,"thread_start_us":114,"threads_started":1}
I20260809 03:25:30.216017 13974 heartbeater.cc:347] Connected to a master server at 127.9.46.254:41961
I20260809 03:25:30.216284 13974 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:30.216802 13974 heartbeater.cc:511] Master 127.9.46.254:41961 requested a full tablet report, sending...
I20260809 03:25:30.218358 13778 ts_manager.cc:194] Registered new tserver with Master: 732fc47fad8b410bb8d844fceffdcc3d (127.9.46.193:37769)
I20260809 03:25:30.219480 13778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:52717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:25:30.230014 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41961
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:38271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:30.343904 13979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:30.344167 13979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:30.344214 13979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:30.348459 13979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:30.348523 13979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:30.348603 13979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:30.353892 13979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:30.354875 13979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:30.355870 13979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:30.365324 13985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:30.365335 13988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:30.365440 13979 server_base.cc:1034] running on GCE node
W20260809 03:25:30.365345 13986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:30.366132 13979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:30.366786 13979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:30.367985 13979 hybrid_clock.cc:648] HybridClock initialized: now 1786245930367957 us; error 38 us; skew 500 ppm
Aug 09 03:25:30 dist-test-slave-65mx krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:30.370949 13979 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:25:30.371768 13979 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:30.371869 13979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:30.372123 13979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:30.374284 13979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c"
format_stamp: "Formatted at 2026-08-09 03:25:30 on dist-test-slave-65mx"
server_key: "df5afab108660dfbc0d63948621a17ec"
server_key_iv: "86b6ac4767a277d27bbdc7819d825964"
server_key_version: "encryptionkey@0"
I20260809 03:25:30.374863 13979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c"
format_stamp: "Formatted at 2026-08-09 03:25:30 on dist-test-slave-65mx"
server_key: "df5afab108660dfbc0d63948621a17ec"
server_key_iv: "86b6ac4767a277d27bbdc7819d825964"
server_key_version: "encryptionkey@0"
I20260809 03:25:30.378676 13979 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 03:25:30.381235 13995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:30.382587 13979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:25:30.382726 13979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c"
format_stamp: "Formatted at 2026-08-09 03:25:30 on dist-test-slave-65mx"
server_key: "df5afab108660dfbc0d63948621a17ec"
server_key_iv: "86b6ac4767a277d27bbdc7819d825964"
server_key_version: "encryptionkey@0"
I20260809 03:25:30.382844 13979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:30.408187 13979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:30.411211 13979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:30.411423 13979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:30.412055 13979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:30.413034 13979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:30.413115 13979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:30.413194 13979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:30.413228 13979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:30.422987 13979 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:38867
I20260809 03:25:30.423029 14113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:38867 every 8 connection(s)
I20260809 03:25:30.423589 13979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:30.427376 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 13979
I20260809 03:25:30.427469 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:30.427745 9403 external_mini_cluster.cc:1428] Setting key f570d09b224c27d1eafc136248303dc6
Aug 09 03:25:30 dist-test-slave-65mx krb5kdc[13731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:30.437968 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.425337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:36585 (local address 127.9.46.254:41961)
0809 03:25:30.425472 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:30.425475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:30.426253 (+ 778us) server_negotiation.cc:407] Connection header received
0809 03:25:30.427684 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:30.427688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:30.427749 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:30.427853 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:30.429218 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.429745 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.430881 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.431106 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.431217 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.434078 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:30.434083 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:30.434085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:30.434111 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:30.435604 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:30.436253 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:30.436256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:30.436257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:30.436332 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:30.436752 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:30.436754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:30.436755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:30.436929 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:30.437020 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:30.437708 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 03:25:30.437845 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:25:30.439126 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.425658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41961 (local address 127.9.46.194:36585)
0809 03:25:30.426104 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:30.426148 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:30.427421 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:30.428026 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:30.428033 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:30.428388 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:30.429074 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.429091 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.429889 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.429892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.430755 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.430764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.431257 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.431261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.431422 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.432079 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:30.432096 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:30.433902 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:30.435730 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:30.435737 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:30.435754 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:30.436151 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:30.436424 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:30.436428 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:30.436430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:30.436657 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:30.437053 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:30.437062 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:30.437396 (+ 334us) client_negotiation.cc:769] Sending connection context
0809 03:25:30.437624 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 03:25:30.437981 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
I20260809 03:25:30.440382 14114 heartbeater.cc:347] Connected to a master server at 127.9.46.254:41961
I20260809 03:25:30.440644 14114 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:30.441087 14114 heartbeater.cc:511] Master 127.9.46.254:41961 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
I20260809 03:25:30.442066 13778 ts_manager.cc:194] Registered new tserver with Master: 8e3d8f9c2f5245b0ad56a356fa917d9c (127.9.46.194:38867)
I20260809 03:25:30.442642 13778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:36585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:25:30.457798 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41961
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:38271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:30.573938 14119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:30.574227 14119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:30.574309 14119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:30.578351 14119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:30.578413 14119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:30.578509 14119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:30.583840 14119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:30.584924 14119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:30.586001 14119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:30.595551 14124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:30.595559 14125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:30.595561 14127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:30.595954 14119 server_base.cc:1034] running on GCE node
I20260809 03:25:30.596369 14119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:30.597065 14119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:30.598264 14119 hybrid_clock.cc:648] HybridClock initialized: now 1786245930598245 us; error 31 us; skew 500 ppm
Aug 09 03:25:30 dist-test-slave-65mx krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:30.601068 14119 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:25:30.601756 14119 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:30.601820 14119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:30.601984 14119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:30.603987 14119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "ab5b3c14e65340a2b469376ee52bf288"
format_stamp: "Formatted at 2026-08-09 03:25:30 on dist-test-slave-65mx"
server_key: "09b5c8e5b5a1ff736b42e9dc08aff4ad"
server_key_iv: "dd9f74365a2c4bd8fa745d6331c0c0be"
server_key_version: "encryptionkey@0"
I20260809 03:25:30.604655 14119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "ab5b3c14e65340a2b469376ee52bf288"
format_stamp: "Formatted at 2026-08-09 03:25:30 on dist-test-slave-65mx"
server_key: "09b5c8e5b5a1ff736b42e9dc08aff4ad"
server_key_iv: "dd9f74365a2c4bd8fa745d6331c0c0be"
server_key_version: "encryptionkey@0"
I20260809 03:25:30.608295 14119 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:25:30.610783 14133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:30.612018 14119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:30.612185 14119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "ab5b3c14e65340a2b469376ee52bf288"
format_stamp: "Formatted at 2026-08-09 03:25:30 on dist-test-slave-65mx"
server_key: "09b5c8e5b5a1ff736b42e9dc08aff4ad"
server_key_iv: "dd9f74365a2c4bd8fa745d6331c0c0be"
server_key_version: "encryptionkey@0"
I20260809 03:25:30.612282 14119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:30.636818 14119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:30.640072 14119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:30.640283 14119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:30.640966 14119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:30.641861 14119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:30.641943 14119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:30.642016 14119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:30.642071 14119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:30.651319 14119 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:38033
I20260809 03:25:30.651335 14253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:38033 every 8 connection(s)
I20260809 03:25:30.651959 14119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:30.655395 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 14119
I20260809 03:25:30.655503 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:30.655802 9403 external_mini_cluster.cc:1428] Setting key 239fe2cf9f8bd5594168c3f62285de87
Aug 09 03:25:30 dist-test-slave-65mx krb5kdc[13731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:30.667309 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.653846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:58609 (local address 127.9.46.254:41961)
0809 03:25:30.654007 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:30.654011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:30.654858 (+ 847us) server_negotiation.cc:407] Connection header received
0809 03:25:30.656371 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:30.656375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:30.656430 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:30.656532 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:30.658204 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.658736 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.659864 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.660076 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.660205 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.663256 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:30.663260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:30.663263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:30.663297 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:30.664981 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:30.665514 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:30.665517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:30.665518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:30.665616 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:30.666007 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:30.666011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:30.666012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:30.666270 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:30.666389 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:30.666969 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 03:25:30.667111 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:25:30.668167 14257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.654222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41961 (local address 127.9.46.195:58609)
0809 03:25:30.654700 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:30.654744 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:30.656141 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:30.656781 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:30.656790 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:30.657329 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:30.658018 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.658039 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.658856 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.658858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.659757 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.659766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.660206 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.660209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.660355 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.661068 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:30.661084 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:30.663030 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:30.665101 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:30.665106 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:30.665122 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:30.665438 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:30.665713 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:30.665716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:30.665717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:30.665896 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:30.666389 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:30.666398 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:30.666674 (+ 276us) client_negotiation.cc:769] Sending connection context
0809 03:25:30.666926 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 03:25:30.667283 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":129,"threads_started":1}
I20260809 03:25:30.669626 14255 heartbeater.cc:347] Connected to a master server at 127.9.46.254:41961
I20260809 03:25:30.669890 14255 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:30.670431 14255 heartbeater.cc:511] Master 127.9.46.254:41961 requested a full tablet report, sending...
I20260809 03:25:30.671455 13778 ts_manager.cc:194] Registered new tserver with Master: ab5b3c14e65340a2b469376ee52bf288 (127.9.46.195:38033)
I20260809 03:25:30.672030 13778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:58609
I20260809 03:25:30.682054 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:25:30.691435 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.684102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60690 (local address 127.9.46.254:41961)
0809 03:25:30.684270 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:30.684279 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:30.684389 (+ 110us) server_negotiation.cc:407] Connection header received
0809 03:25:30.684477 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:30.684479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:30.684521 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:30.684640 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:30.685307 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.685810 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.686696 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.686919 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.687038 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.688534 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:30.688600 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:30.688606 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:30.688639 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:30.689865 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:30.690278 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:30.690281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:30.690282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:30.690361 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:30.690596 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:30.690598 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:30.690599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:30.690819 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:30.690920 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:30.691126 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 03:25:30.691225 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 03:25:30.694556 13778 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:30.697618 13778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:30.712190 14267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.706009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59394 (local address 127.9.46.193:37769)
0809 03:25:30.707014 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:30.707021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:30.707050 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:25:30.707135 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:30.707142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:30.707391 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:30.707548 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:30.707892 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.708965 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.710700 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.711409 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.711521 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.711674 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:30.711900 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 03:25:30.712006 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":892,"thread_start_us":104,"threads_started":1}
I20260809 03:25:30.712181 14266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.705792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:37769 (local address 127.0.0.1:59394)
0809 03:25:30.706604 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:30.706666 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:30.706789 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:30.707563 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:30.707567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:30.707589 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:30.707754 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.707762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.709091 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.709096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.710548 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.710559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.711544 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.711547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.711671 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.711733 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 03:25:30.711844 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 03:25:30.711995 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":52,"thread_start_us":102,"threads_started":1}
I20260809 03:25:30.712604 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.706582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44286 (local address 127.9.46.194:38867)
0809 03:25:30.707483 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:30.707497 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:30.707746 (+ 249us) server_negotiation.cc:407] Connection header received
0809 03:25:30.707864 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:30.707869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:30.708017 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:30.708184 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:30.708360 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.709680 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.710856 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.711598 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.711699 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.711841 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:30.712268 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 03:25:30.712415 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":116,"threads_started":1}
I20260809 03:25:30.712790 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.706094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:38033 (local address 127.0.0.1:51134)
0809 03:25:30.707110 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:30.707124 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:30.707213 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:30.708498 (+ 1285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:30.708501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:30.708518 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:30.708637 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.708643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.709974 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.709977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.711076 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.711087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.712334 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.712339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.712464 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.712485 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:25:30.712574 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 03:25:30.712647 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"mutex_wait_us":25,"thread_start_us":121,"threads_started":1}
I20260809 03:25:30.713003 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.706156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:38867 (local address 127.0.0.1:44286)
0809 03:25:30.707625 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:30.707639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:30.707754 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:30.708151 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:30.708154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:30.708172 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:30.708270 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.708276 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.709822 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.709825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.710728 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.710738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.711902 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.711905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.712015 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.712036 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:25:30.712104 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 03:25:30.712159 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1405,"mutex_wait_us":45,"thread_start_us":78,"threads_started":1}
I20260809 03:25:30.713483 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.706582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51134 (local address 127.9.46.195:38033)
0809 03:25:30.707901 (+ 1319us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:30.707914 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:30.707935 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:25:30.708006 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:30.708010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:30.708185 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:30.708342 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:30.708762 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.709863 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.711213 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.711997 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.712107 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.712256 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:30.713210 (+ 954us) server_negotiation.cc:299] Negotiation successful
0809 03:25:30.713313 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1215,"thread_start_us":125,"threads_started":1}
I20260809 03:25:30.715706 13907 tablet_service.cc:1467] Processing CreateTablet for tablet cf489ca372614e67a53603939f269194 (DEFAULT_TABLE table=test-table [id=40d28f57ac9e41db911f0dc29e39620e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:30.715912 14044 tablet_service.cc:1467] Processing CreateTablet for tablet cf489ca372614e67a53603939f269194 (DEFAULT_TABLE table=test-table [id=40d28f57ac9e41db911f0dc29e39620e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:30.716006 14184 tablet_service.cc:1467] Processing CreateTablet for tablet cf489ca372614e67a53603939f269194 (DEFAULT_TABLE table=test-table [id=40d28f57ac9e41db911f0dc29e39620e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:30.716915 13907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf489ca372614e67a53603939f269194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:30.716919 14184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf489ca372614e67a53603939f269194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:30.716979 14044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf489ca372614e67a53603939f269194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:30.723521 14272 tablet_bootstrap.cc:492] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: Bootstrap starting.
I20260809 03:25:30.724252 14273 tablet_bootstrap.cc:492] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d: Bootstrap starting.
I20260809 03:25:30.724963 14275 tablet_bootstrap.cc:492] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: Bootstrap starting.
I20260809 03:25:30.726758 14275 tablet_bootstrap.cc:654] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:30.726784 14273 tablet_bootstrap.cc:654] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:30.726886 14272 tablet_bootstrap.cc:654] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:30.727818 14273 log.cc:824] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:30.728056 14275 log.cc:824] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:30.728209 14272 log.cc:824] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:30.730060 14275 tablet_bootstrap.cc:492] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: No bootstrap required, opened a new log
I20260809 03:25:30.730203 14273 tablet_bootstrap.cc:492] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d: No bootstrap required, opened a new log
I20260809 03:25:30.730209 14272 tablet_bootstrap.cc:492] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: No bootstrap required, opened a new log
I20260809 03:25:30.730306 14275 ts_tablet_manager.cc:1397] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:25:30.730399 14273 ts_tablet_manager.cc:1397] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:25:30.730399 14272 ts_tablet_manager.cc:1397] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 03:25:30.733330 14275 raft_consensus.cc:348] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732fc47fad8b410bb8d844fceffdcc3d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37769 } } peers { permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 } } peers { permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 } }
I20260809 03:25:30.733599 14275 raft_consensus.cc:374] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:30.733657 14275 raft_consensus.cc:729] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab5b3c14e65340a2b469376ee52bf288, State: Initialized, Role: FOLLOWER
I20260809 03:25:30.734113 14275 consensus_queue.cc:260] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732fc47fad8b410bb8d844fceffdcc3d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37769 } } peers { permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 } } peers { permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 } }
I20260809 03:25:30.734391 14272 raft_consensus.cc:348] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732fc47fad8b410bb8d844fceffdcc3d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37769 } } peers { permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 } } peers { permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 } }
I20260809 03:25:30.734649 14272 raft_consensus.cc:374] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:30.734709 14272 raft_consensus.cc:729] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e3d8f9c2f5245b0ad56a356fa917d9c, State: Initialized, Role: FOLLOWER
I20260809 03:25:30.734812 14273 raft_consensus.cc:348] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732fc47fad8b410bb8d844fceffdcc3d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37769 } } peers { permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 } } peers { permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 } }
I20260809 03:25:30.735052 14273 raft_consensus.cc:374] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:30.735101 14273 raft_consensus.cc:729] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732fc47fad8b410bb8d844fceffdcc3d, State: Initialized, Role: FOLLOWER
I20260809 03:25:30.735253 14275 ts_tablet_manager.cc:1428] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:25:30.735175 14272 consensus_queue.cc:260] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732fc47fad8b410bb8d844fceffdcc3d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37769 } } peers { permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 } } peers { permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 } }
I20260809 03:25:30.735482 14273 consensus_queue.cc:260] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732fc47fad8b410bb8d844fceffdcc3d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37769 } } peers { permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 } } peers { permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 } }
I20260809 03:25:30.735643 14255 heartbeater.cc:503] Master 127.9.46.254:41961 was elected leader, sending a full tablet report...
I20260809 03:25:30.736119 13974 heartbeater.cc:503] Master 127.9.46.254:41961 was elected leader, sending a full tablet report...
I20260809 03:25:30.736438 14273 ts_tablet_manager.cc:1428] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 03:25:30.737273 14114 heartbeater.cc:503] Master 127.9.46.254:41961 was elected leader, sending a full tablet report...
I20260809 03:25:30.737293 14272 ts_tablet_manager.cc:1428] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260809 03:25:30.902932 14256 tablet.cc:2369] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:25:30.924706 14115 tablet.cc:2369] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:30.936272 14280 raft_consensus.cc:482] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:30.936527 14280 raft_consensus.cc:504] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732fc47fad8b410bb8d844fceffdcc3d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37769 } } peers { permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 } } peers { permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 } }
I20260809 03:25:30.937768 14280 leader_election.cc:288] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab5b3c14e65340a2b469376ee52bf288 (127.9.46.195:38033), 8e3d8f9c2f5245b0ad56a356fa917d9c (127.9.46.194:38867)
I20260809 03:25:30.942229 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.938618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:40713 (local address 127.9.46.194:38867)
0809 03:25:30.938737 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:30.938744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:30.938762 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:25:30.938852 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:30.938855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:30.938905 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:30.939006 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:30.939327 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.940028 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.941192 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.941610 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.941741 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.941793 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:30.942035 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 03:25:30.942107 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:25:30.942231 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.938344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:37877 (local address 127.9.46.195:38033)
0809 03:25:30.938516 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:30.938521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:30.938742 (+ 221us) server_negotiation.cc:407] Connection header received
0809 03:25:30.938936 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:30.938939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:30.938994 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:30.939061 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:30.939412 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.939906 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.941192 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.941609 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.941743 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.941800 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:30.942037 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 03:25:30.942110 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:25:30.942257 14283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.938095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:38867 (local address 127.9.46.193:40713)
0809 03:25:30.938589 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:30.938605 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:30.938749 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:30.939009 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:30.939013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:30.939036 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:30.939178 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.939184 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.940136 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.940138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.941063 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.941073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.941736 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.941740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.941891 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.941922 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 03:25:30.942015 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 03:25:30.942088 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":60,"thread_start_us":152,"threads_started":1}
I20260809 03:25:30.942263 14282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:30.938128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:38033 (local address 127.9.46.193:37877)
0809 03:25:30.938646 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:30.938660 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:30.938806 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:30.939170 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:30.939172 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:30.939187 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:30.939306 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.939312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.940024 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.940028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.941053 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:30.941063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:30.941743 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:30.941747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:30.941901 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:30.941930 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 03:25:30.942022 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 03:25:30.942093 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":31,"thread_start_us":153,"threads_started":1}
I20260809 03:25:30.943068 14064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf489ca372614e67a53603939f269194" candidate_uuid: "732fc47fad8b410bb8d844fceffdcc3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" is_pre_election: true
I20260809 03:25:30.943187 14205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf489ca372614e67a53603939f269194" candidate_uuid: "732fc47fad8b410bb8d844fceffdcc3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab5b3c14e65340a2b469376ee52bf288" is_pre_election: true
I20260809 03:25:30.943418 14205 raft_consensus.cc:2454] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 732fc47fad8b410bb8d844fceffdcc3d in term 0.
I20260809 03:25:30.943418 14064 raft_consensus.cc:2454] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 732fc47fad8b410bb8d844fceffdcc3d in term 0.
I20260809 03:25:30.943931 13857 leader_election.cc:302] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 732fc47fad8b410bb8d844fceffdcc3d, ab5b3c14e65340a2b469376ee52bf288; no voters:
I20260809 03:25:30.944248 14280 raft_consensus.cc:2781] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:30.944340 14280 raft_consensus.cc:482] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:30.944388 14280 raft_consensus.cc:3037] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:30.945509 14280 raft_consensus.cc:504] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732fc47fad8b410bb8d844fceffdcc3d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37769 } } peers { permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 } } peers { permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 } }
I20260809 03:25:30.945837 14280 leader_election.cc:288] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [CANDIDATE]: Term 1 election: Requested vote from peers ab5b3c14e65340a2b469376ee52bf288 (127.9.46.195:38033), 8e3d8f9c2f5245b0ad56a356fa917d9c (127.9.46.194:38867)
I20260809 03:25:30.946157 14205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf489ca372614e67a53603939f269194" candidate_uuid: "732fc47fad8b410bb8d844fceffdcc3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab5b3c14e65340a2b469376ee52bf288"
I20260809 03:25:30.946188 14064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf489ca372614e67a53603939f269194" candidate_uuid: "732fc47fad8b410bb8d844fceffdcc3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c"
I20260809 03:25:30.946295 14064 raft_consensus.cc:3037] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:30.946290 14205 raft_consensus.cc:3037] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288 [term 0 FOLLOWER]: Advancing to term 1
W20260809 03:25:30.947198 13975 tablet.cc:2369] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:30.947592 14064 raft_consensus.cc:2454] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 732fc47fad8b410bb8d844fceffdcc3d in term 1.
I20260809 03:25:30.947592 14205 raft_consensus.cc:2454] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 732fc47fad8b410bb8d844fceffdcc3d in term 1.
I20260809 03:25:30.947908 13860 leader_election.cc:302] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 732fc47fad8b410bb8d844fceffdcc3d, 8e3d8f9c2f5245b0ad56a356fa917d9c; no voters:
I20260809 03:25:30.948114 14280 raft_consensus.cc:2781] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:30.948319 14280 raft_consensus.cc:686] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 1 LEADER]: Becoming Leader. State: Replica: 732fc47fad8b410bb8d844fceffdcc3d, State: Running, Role: LEADER
I20260809 03:25:30.948673 14280 consensus_queue.cc:237] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732fc47fad8b410bb8d844fceffdcc3d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37769 } } peers { permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 } } peers { permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 } }
I20260809 03:25:30.951736 13777 catalog_manager.cc:5495] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 732fc47fad8b410bb8d844fceffdcc3d (127.9.46.193). New cstate: current_term: 1 leader_uuid: "732fc47fad8b410bb8d844fceffdcc3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732fc47fad8b410bb8d844fceffdcc3d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 } health_report { overall_health: UNKNOWN } } }
I20260809 03:25:31.004104 14267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.000880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.9.46.193:37769)
0809 03:25:31.001015 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.001019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.001055 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:25:31.001141 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.001143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.001205 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.001289 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:31.001515 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:31.002011 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:31.002816 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:31.003020 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:31.003097 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:31.003376 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:31.003782 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:31.003871 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.003942 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.003993 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:25:31.009956 14205 raft_consensus.cc:1261] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288 [term 1 FOLLOWER]: Refusing update from remote peer 732fc47fad8b410bb8d844fceffdcc3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:31.009958 14064 raft_consensus.cc:1261] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c [term 1 FOLLOWER]: Refusing update from remote peer 732fc47fad8b410bb8d844fceffdcc3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:31.010670 14284 consensus_queue.cc:1035] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e3d8f9c2f5245b0ad56a356fa917d9c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 38867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:31.010876 14280 consensus_queue.cc:1035] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab5b3c14e65340a2b469376ee52bf288" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:31.017954 14287 mvcc.cc:205] Tried to move back new op lower bound from 7316463333409042432 to 7316463333168525312. Current Snapshot: MvccSnapshot[applied={T|T < 7316463333409042432}]
I20260809 03:25:31.018165 14289 mvcc.cc:205] Tried to move back new op lower bound from 7316463333409042432 to 7316463333168525312. Current Snapshot: MvccSnapshot[applied={T|T < 7316463333409042432}]
I20260809 03:25:31.021245 14288 mvcc.cc:205] Tried to move back new op lower bound from 7316463333409042432 to 7316463333168525312. Current Snapshot: MvccSnapshot[applied={T|T < 7316463333409042432}]
I20260809 03:25:31.027101 13777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:31.027292 13777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:31.029973 13777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b121a3d53b7248e2b8b7cfc7964c9f93: Sending DeleteTablet for 3 replicas of tablet cf489ca372614e67a53603939f269194
I20260809 03:25:31.030782 13907 tablet_service.cc:1514] Processing DeleteTablet for tablet cf489ca372614e67a53603939f269194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:31 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:59394
I20260809 03:25:31.030856 14184 tablet_service.cc:1514] Processing DeleteTablet for tablet cf489ca372614e67a53603939f269194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:31 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:51134
I20260809 03:25:31.030916 14044 tablet_service.cc:1514] Processing DeleteTablet for tablet cf489ca372614e67a53603939f269194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:31 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:44286
I20260809 03:25:31.031360 14298 tablet_replica.cc:330] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: stopping tablet replica
I20260809 03:25:31.031360 14299 tablet_replica.cc:330] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: stopping tablet replica
I20260809 03:25:31.031708 14298 raft_consensus.cc:2229] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:31.031842 14297 tablet_replica.cc:330] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d: stopping tablet replica
I20260809 03:25:31.032053 14298 raft_consensus.cc:2258] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:31.032217 14299 raft_consensus.cc:2229] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:31.032317 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 13841
I20260809 03:25:31.032269 14297 raft_consensus.cc:2229] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:31.032528 14299 raft_consensus.cc:2258] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:31.032768 14297 raft_consensus.cc:2258] T cf489ca372614e67a53603939f269194 P 732fc47fad8b410bb8d844fceffdcc3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:31.034204 14299 ts_tablet_manager.cc:1905] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:31.035035 14298 ts_tablet_manager.cc:1905] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:31.037406 14299 ts_tablet_manager.cc:1918] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:31.037516 14299 log.cc:1196] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/cf489ca372614e67a53603939f269194
I20260809 03:25:31.037863 14299 ts_tablet_manager.cc:1939] T cf489ca372614e67a53603939f269194 P 8e3d8f9c2f5245b0ad56a356fa917d9c: Deleting consensus metadata
I20260809 03:25:31.038242 14298 ts_tablet_manager.cc:1918] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:31.038403 14298 log.cc:1196] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/cf489ca372614e67a53603939f269194
I20260809 03:25:31.038803 14298 ts_tablet_manager.cc:1939] T cf489ca372614e67a53603939f269194 P ab5b3c14e65340a2b469376ee52bf288: Deleting consensus metadata
I20260809 03:25:31.039239 13763 catalog_manager.cc:4841] TS 8e3d8f9c2f5245b0ad56a356fa917d9c (127.9.46.194:38867): tablet cf489ca372614e67a53603939f269194 (table test-table [id=40d28f57ac9e41db911f0dc29e39620e]) successfully deleted
I20260809 03:25:31.039990 13759 catalog_manager.cc:4841] TS ab5b3c14e65340a2b469376ee52bf288 (127.9.46.195:38033): tablet cf489ca372614e67a53603939f269194 (table test-table [id=40d28f57ac9e41db911f0dc29e39620e]) successfully deleted
W20260809 03:25:31.047133 13762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:37769 (error 108)
I20260809 03:25:31.047991 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 13979
I20260809 03:25:31.048250 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.047937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:37769 (local address 127.0.0.1:59422)
0809 03:25:31.048078 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:31.048152 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 03:25:31.048929 13762 catalog_manager.cc:4584] TS 732fc47fad8b410bb8d844fceffdcc3d (127.9.46.193:37769): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf489ca372614e67a53603939f269194: Network error: Client connection negotiation failed: client connection to 127.9.46.193:37769: connect: Connection refused (error 111)
I20260809 03:25:31.055781 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 14119
I20260809 03:25:31.063704 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 13744
2026-08-09T03:25:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2568 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T03:25:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:31Z Disabled control of system clock
I20260809 03:25:31.097713 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34405
--webserver_interface=127.9.46.254
--webserver_port=0
--builtin_ntp_servers=127.9.46.212:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:34405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 03:25:31.209213 14304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:31.209486 14304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:31.209522 14304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:31.213587 14304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:31.213645 14304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:31.213661 14304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:31.213678 14304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:31.213730 14304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:31.218534 14304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:34405
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:31.219627 14304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:31.221050 14304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:31.229532 14310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:31.229517 14313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:31.229578 14309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:31.229979 14304 server_base.cc:1034] running on GCE node
I20260809 03:25:31.230502 14304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:31.231626 14304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:31.232787 14304 hybrid_clock.cc:648] HybridClock initialized: now 1786245931232761 us; error 41 us; skew 500 ppm
I20260809 03:25:31.234678 14304 webserver.cc:460] Webserver started at http://127.9.46.254:38723/ using document root <none> and password file <none>
I20260809 03:25:31.235267 14304 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:31.235332 14304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:31.235559 14304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:31.237731 14304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "f47be19d720a4089a5f92b16b47c79e1"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "29e808748381c98ac73c6d274b29a735"
server_key_iv: "fa7c36fc2405795bd95d62d70fa743ee"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.238335 14304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "f47be19d720a4089a5f92b16b47c79e1"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "29e808748381c98ac73c6d274b29a735"
server_key_iv: "fa7c36fc2405795bd95d62d70fa743ee"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.242025 14304 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:25:31.244464 14320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:31.245695 14304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:31.245823 14304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "f47be19d720a4089a5f92b16b47c79e1"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "29e808748381c98ac73c6d274b29a735"
server_key_iv: "fa7c36fc2405795bd95d62d70fa743ee"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.245937 14304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:31.250555 14304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:31.251158 14304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:31.251348 14304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:31.258579 14304 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:34405
I20260809 03:25:31.258605 14375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:34405 every 8 connection(s)
I20260809 03:25:31.259531 14304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:31.262779 14376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:31.263675 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 14304
I20260809 03:25:31.263790 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:31.264092 9403 external_mini_cluster.cc:1428] Setting key 03c2225ea9abe3a0ed16470d61038d1f
I20260809 03:25:31.269363 14376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1: Bootstrap starting.
I20260809 03:25:31.269925 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.265608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38574 (local address 127.9.46.254:34405)
0809 03:25:31.266257 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.266276 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.266310 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:25:31.267011 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.267028 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.267374 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.267716 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.268600 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.268607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.268621 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.268640 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.268696 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.268714 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.268869 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.268942 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.269188 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":134,"threads_started":1}
I20260809 03:25:31.271811 14376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:31.272650 14376 log.cc:824] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:31.274466 14376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1: No bootstrap required, opened a new log
I20260809 03:25:31.278203 14376 raft_consensus.cc:348] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47be19d720a4089a5f92b16b47c79e1" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34405 } }
I20260809 03:25:31.278416 14376 raft_consensus.cc:374] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:31.278494 14376 raft_consensus.cc:729] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f47be19d720a4089a5f92b16b47c79e1, State: Initialized, Role: FOLLOWER
I20260809 03:25:31.278887 14376 consensus_queue.cc:260] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47be19d720a4089a5f92b16b47c79e1" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34405 } }
I20260809 03:25:31.279028 14376 raft_consensus.cc:388] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:31.279096 14376 raft_consensus.cc:482] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:31.279227 14376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:31.280123 14376 raft_consensus.cc:504] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47be19d720a4089a5f92b16b47c79e1" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34405 } }
I20260809 03:25:31.280462 14376 leader_election.cc:302] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f47be19d720a4089a5f92b16b47c79e1; no voters:
I20260809 03:25:31.280797 14376 leader_election.cc:288] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:31.280920 14381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:31.281150 14381 raft_consensus.cc:686] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [term 1 LEADER]: Becoming Leader. State: Replica: f47be19d720a4089a5f92b16b47c79e1, State: Running, Role: LEADER
I20260809 03:25:31.281445 14381 consensus_queue.cc:237] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47be19d720a4089a5f92b16b47c79e1" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34405 } }
I20260809 03:25:31.282043 14376 sys_catalog.cc:564] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:31.283121 14382 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f47be19d720a4089a5f92b16b47c79e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47be19d720a4089a5f92b16b47c79e1" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34405 } } }
I20260809 03:25:31.283262 14382 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:31.283515 14383 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f47be19d720a4089a5f92b16b47c79e1. Latest consensus state: current_term: 1 leader_uuid: "f47be19d720a4089a5f92b16b47c79e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47be19d720a4089a5f92b16b47c79e1" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34405 } } }
I20260809 03:25:31.283605 14383 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:31.284592 14390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:31.287145 14390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:31.292886 14390 catalog_manager.cc:1347] Generated new cluster ID: 0bf019fe0d7c4fdb9cf3b38beb6dacf1
I20260809 03:25:31.292969 14390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:31.305048 14390 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:31.305377 14390 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:31.312532 14390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1: Generated new TSK 0
I20260809 03:25:31.313225 14390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:25:31.316409 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34405
--builtin_ntp_servers=127.9.46.212:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 03:25:31.431809 14401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:31.432075 14401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:31.432147 14401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:31.436249 14401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:31.436307 14401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:31.436386 14401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:31.441279 14401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:31.442377 14401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:31.443820 14401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:31.453431 14407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:31.453444 14406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:31.453507 14401 server_base.cc:1034] running on GCE node
W20260809 03:25:31.453444 14409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:31.454049 14401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:31.454708 14401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:31.455895 14401 hybrid_clock.cc:648] HybridClock initialized: now 1786245931455874 us; error 30 us; skew 500 ppm
I20260809 03:25:31.457850 14401 webserver.cc:460] Webserver started at http://127.9.46.193:40297/ using document root <none> and password file <none>
I20260809 03:25:31.458441 14401 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:31.458541 14401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:31.458818 14401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:31.460983 14401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "80029a5c5fed481498ff14bf4951f395"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "33d1245ca323e05327e9f161f186b355"
server_key_iv: "b6bda7b26f863a4f465a08680e3f7a1b"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.461498 14401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "80029a5c5fed481498ff14bf4951f395"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "33d1245ca323e05327e9f161f186b355"
server_key_iv: "b6bda7b26f863a4f465a08680e3f7a1b"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.465042 14401 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:25:31.467324 14415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:31.468626 14401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:31.468757 14401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "80029a5c5fed481498ff14bf4951f395"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "33d1245ca323e05327e9f161f186b355"
server_key_iv: "b6bda7b26f863a4f465a08680e3f7a1b"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.468866 14401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:31.473404 14401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:31.474009 14401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:31.474197 14401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:31.474750 14401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:31.475626 14401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:31.475682 14401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:31.475747 14401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:31.475780 14401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:31.484844 14401 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:34569
I20260809 03:25:31.484866 14531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:34569 every 8 connection(s)
I20260809 03:25:31.485742 14401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:31.491761 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.487600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:54965 (local address 127.9.46.254:34405)
0809 03:25:31.487737 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.487741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.488878 (+ 1137us) server_negotiation.cc:407] Connection header received
0809 03:25:31.490257 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.490260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.490297 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.490372 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.491379 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.491382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.491384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.491395 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.491413 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.491421 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.491496 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.491617 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.491656 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:25:31.492678 14534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.487998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34405 (local address 127.9.46.193:54965)
0809 03:25:31.488732 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:31.488775 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:31.490021 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:31.490530 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:31.490536 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:31.490620 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:31.491136 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:25:31.491137 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:31.491175 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:25:31.491239 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:31.491496 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:31.491503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:31.491506 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:25:31.491633 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 03:25:31.491841 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":149,"threads_started":1}
I20260809 03:25:31.493731 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 14401
I20260809 03:25:31.493861 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:31.493937 14532 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34405
I20260809 03:25:31.494182 14532 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:31.494199 9403 external_mini_cluster.cc:1428] Setting key 19fb0e768909ca790dc3db4bdbac997f
I20260809 03:25:31.494611 14532 heartbeater.cc:511] Master 127.9.46.254:34405 requested a full tablet report, sending...
I20260809 03:25:31.496098 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34405
--builtin_ntp_servers=127.9.46.212:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 03:25:31.496105 14337 ts_manager.cc:194] Registered new tserver with Master: 80029a5c5fed481498ff14bf4951f395 (127.9.46.193:34569)
W20260809 03:25:31.607575 14535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:31.607858 14535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:31.607901 14535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:31.611989 14535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:31.612057 14535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:31.612191 14535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:31.617041 14535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:31.618113 14535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:31.619441 14535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:25:31.629086 14535 server_base.cc:1034] running on GCE node
W20260809 03:25:31.629117 14541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:31.629030 14542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:31.629241 14544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:31.629683 14535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:31.630352 14535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:31.631544 14535 hybrid_clock.cc:648] HybridClock initialized: now 1786245931631511 us; error 45 us; skew 500 ppm
I20260809 03:25:31.633477 14535 webserver.cc:460] Webserver started at http://127.9.46.194:33955/ using document root <none> and password file <none>
I20260809 03:25:31.634063 14535 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:31.634166 14535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:31.634405 14535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:31.636482 14535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "cd0289ff5c404c2282feb23a53423603"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "fcfcb1c39c77c30862010d1aa6d32b09"
server_key_iv: "6d7d4cec0c002859d6f7d1b0e913eb9c"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.637144 14535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "cd0289ff5c404c2282feb23a53423603"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "fcfcb1c39c77c30862010d1aa6d32b09"
server_key_iv: "6d7d4cec0c002859d6f7d1b0e913eb9c"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.640849 14535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:31.643342 14550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:31.644629 14535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:31.644768 14535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "cd0289ff5c404c2282feb23a53423603"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "fcfcb1c39c77c30862010d1aa6d32b09"
server_key_iv: "6d7d4cec0c002859d6f7d1b0e913eb9c"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.644884 14535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:31.649526 14535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:31.650167 14535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:31.650352 14535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:31.650906 14535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:31.651808 14535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:31.651881 14535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:31.651948 14535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:31.651983 14535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:31.664428 14535 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:41145
I20260809 03:25:31.664469 14670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:41145 every 8 connection(s)
I20260809 03:25:31.666306 14535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:31.673240 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.668437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:51381 (local address 127.9.46.254:34405)
0809 03:25:31.668654 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.668663 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.669775 (+ 1112us) server_negotiation.cc:407] Connection header received
0809 03:25:31.671101 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.671106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.671170 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.671286 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.672637 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.672643 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.672649 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.672667 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.672694 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.672708 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.672819 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.672976 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.673032 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:25:31.673348 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 14535
I20260809 03:25:31.673493 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:31.673811 9403 external_mini_cluster.cc:1428] Setting key d6d69be9b65de922482b27308cf90123
I20260809 03:25:31.674358 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.668872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34405 (local address 127.9.46.194:51381)
0809 03:25:31.669603 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:31.669655 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:31.670879 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:31.671486 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:31.671493 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:31.671585 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:31.672320 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:25:31.672321 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:31.672361 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:25:31.672459 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:31.672816 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:31.672826 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:31.672831 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:25:31.672979 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 03:25:31.673252 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":139,"threads_started":1}
I20260809 03:25:31.675597 14671 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34405
I20260809 03:25:31.676086 14671 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:31.676785 14671 heartbeater.cc:511] Master 127.9.46.254:34405 requested a full tablet report, sending...
I20260809 03:25:31.677114 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34405
--builtin_ntp_servers=127.9.46.212:38961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 03:25:31.678077 14337 ts_manager.cc:194] Registered new tserver with Master: cd0289ff5c404c2282feb23a53423603 (127.9.46.194:41145)
W20260809 03:25:31.792083 14674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:31.792341 14674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:31.792382 14674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:31.796520 14674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:31.796653 14674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:31.796743 14674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:31.801669 14674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:31.802793 14674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:31.804258 14674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:31.814059 14679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:31.814220 14682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:31.814328 14674 server_base.cc:1034] running on GCE node
W20260809 03:25:31.814067 14680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:31.814893 14674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:31.815537 14674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:31.816735 14674 hybrid_clock.cc:648] HybridClock initialized: now 1786245931816707 us; error 48 us; skew 500 ppm
I20260809 03:25:31.818590 14674 webserver.cc:460] Webserver started at http://127.9.46.195:38711/ using document root <none> and password file <none>
I20260809 03:25:31.819172 14674 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:31.819269 14674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:31.819507 14674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:31.821604 14674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "dd99dc07bc6949ea922eeda4e57b0249"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "9ff6084559cf93e1a902205ba305b3e0"
server_key_iv: "c5f496d24681c062c4da9069f60ff72c"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.822182 14674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "dd99dc07bc6949ea922eeda4e57b0249"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "9ff6084559cf93e1a902205ba305b3e0"
server_key_iv: "c5f496d24681c062c4da9069f60ff72c"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.826094 14674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:25:31.828688 14688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:31.830004 14674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:31.830147 14674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "dd99dc07bc6949ea922eeda4e57b0249"
format_stamp: "Formatted at 2026-08-09 03:25:31 on dist-test-slave-65mx"
server_key: "9ff6084559cf93e1a902205ba305b3e0"
server_key_iv: "c5f496d24681c062c4da9069f60ff72c"
server_key_version: "encryptionkey@0"
I20260809 03:25:31.830261 14674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:31.834972 14674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:31.835624 14674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:31.835809 14674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:31.836364 14674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:31.837338 14674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:31.837386 14674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:31.837453 14674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:31.837494 14674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:31.846161 14674 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:37921
I20260809 03:25:31.846199 14808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:37921 every 8 connection(s)
I20260809 03:25:31.847101 14674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:31.853353 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.848990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:35499 (local address 127.9.46.254:34405)
0809 03:25:31.849155 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.849159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.850330 (+ 1171us) server_negotiation.cc:407] Connection header received
0809 03:25:31.851641 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.851645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.851692 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.851782 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.852861 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.852864 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.852867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.852880 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.852908 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.852916 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.853017 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.853168 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.853215 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:25:31.854167 14811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.849471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34405 (local address 127.9.46.195:35499)
0809 03:25:31.850185 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:31.850232 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:31.851430 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:31.851941 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:31.851947 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:31.852031 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:31.852595 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:25:31.852598 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:31.852649 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:25:31.852714 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:31.853001 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:31.853010 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:31.853016 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:25:31.853213 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 03:25:31.853445 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":139,"threads_started":1}
I20260809 03:25:31.854687 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 14674
I20260809 03:25:31.854840 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:31.855212 9403 external_mini_cluster.cc:1428] Setting key b5dc226f73e5b9cb83280a71892f99ca
I20260809 03:25:31.855398 14809 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34405
I20260809 03:25:31.855645 14809 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:31.856092 14809 heartbeater.cc:511] Master 127.9.46.254:34405 requested a full tablet report, sending...
I20260809 03:25:31.857239 14336 ts_manager.cc:194] Registered new tserver with Master: dd99dc07bc6949ea922eeda4e57b0249 (127.9.46.195:37921)
I20260809 03:25:31.869341 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:25:31.873092 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.871324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38588 (local address 127.9.46.254:34405)
0809 03:25:31.871462 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.871466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.871691 (+ 225us) server_negotiation.cc:407] Connection header received
0809 03:25:31.871816 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.871818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.871852 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.871921 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.872620 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.872624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.872626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.872650 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.872670 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.872678 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.872749 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.872881 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.872916 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:25:31.876003 14336 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:31.879168 14336 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:31.891170 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.887963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:41145 (local address 127.0.0.1:57214)
0809 03:25:31.888939 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:31.888997 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:31.889105 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:31.889638 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:31.889644 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:31.889714 (+ 70us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:31.890315 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:25:31.890319 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:31.890340 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:25:31.890411 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:31.890802 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:31.890811 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:31.890817 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:25:31.890928 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 03:25:31.891007 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":140,"threads_started":1}
I20260809 03:25:31.891705 14821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.887861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.9.46.195:37921)
0809 03:25:31.888496 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.888511 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.889089 (+ 578us) server_negotiation.cc:407] Connection header received
0809 03:25:31.889279 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.889284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.889484 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.889598 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.891047 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.891052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.891060 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.891068 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.891099 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.891114 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.891244 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.891437 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.891507 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":116,"threads_started":1}
I20260809 03:25:31.891744 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.888136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.9.46.194:41145)
0809 03:25:31.888609 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.888622 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.889214 (+ 592us) server_negotiation.cc:407] Connection header received
0809 03:25:31.889278 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.889283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.889484 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.889610 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.890582 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.890591 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.890606 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.890623 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.890667 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.890709 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.890829 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.890940 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.891011 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":113,"threads_started":1}
I20260809 03:25:31.892017 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.888600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:34569 (local address 127.0.0.1:42220)
0809 03:25:31.889283 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:31.889296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:31.889381 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:31.890458 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:31.890463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:31.890483 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:31.891345 (+ 862us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:25:31.891347 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:31.891358 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:25:31.891389 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:31.891774 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:31.891780 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:31.891785 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:25:31.891858 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 03:25:31.891893 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":103,"threads_started":1}
I20260809 03:25:31.892355 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.887711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:37921 (local address 127.0.0.1:51554)
0809 03:25:31.888939 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:31.888997 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:31.889155 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:31.890346 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:31.890349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:31.890364 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:31.890812 (+ 448us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:25:31.890814 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:31.890827 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:25:31.890863 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:31.891217 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:31.891220 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:31.891321 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 03:25:31.892207 (+ 886us) client_negotiation.cc:240] Negotiation successful
0809 03:25:31.892245 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"thread_start_us":84,"threads_started":1}
I20260809 03:25:31.892521 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.888823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.9.46.193:34569)
0809 03:25:31.889307 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.889325 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.889354 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:25:31.889501 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.889506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.889690 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.889821 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.891524 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.891530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.891540 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.891552 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.891596 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.891629 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.891731 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.892299 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.892353 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":141,"threads_started":1}
I20260809 03:25:31.894645 14592 tablet_service.cc:1467] Processing CreateTablet for tablet 72d6d81954b44b60be4b8c2bc25180e0 (DEFAULT_TABLE table=test-table [id=bd11f67b71974da0b0e88e42c20c3ad7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:31.895013 14740 tablet_service.cc:1467] Processing CreateTablet for tablet 72d6d81954b44b60be4b8c2bc25180e0 (DEFAULT_TABLE table=test-table [id=bd11f67b71974da0b0e88e42c20c3ad7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:31.895015 14464 tablet_service.cc:1467] Processing CreateTablet for tablet 72d6d81954b44b60be4b8c2bc25180e0 (DEFAULT_TABLE table=test-table [id=bd11f67b71974da0b0e88e42c20c3ad7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:31.895844 14592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d6d81954b44b60be4b8c2bc25180e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:31.896051 14464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d6d81954b44b60be4b8c2bc25180e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:31.896051 14740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d6d81954b44b60be4b8c2bc25180e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:31.902103 14826 tablet_bootstrap.cc:492] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: Bootstrap starting.
I20260809 03:25:31.903673 14828 tablet_bootstrap.cc:492] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395: Bootstrap starting.
I20260809 03:25:31.903673 14827 tablet_bootstrap.cc:492] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: Bootstrap starting.
I20260809 03:25:31.904726 14826 tablet_bootstrap.cc:654] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:31.905468 14826 log.cc:824] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:31.905855 14827 tablet_bootstrap.cc:654] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:31.905855 14828 tablet_bootstrap.cc:654] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:31.906694 14827 log.cc:824] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:31.906694 14828 log.cc:824] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:31.907438 14826 tablet_bootstrap.cc:492] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: No bootstrap required, opened a new log
I20260809 03:25:31.907632 14826 ts_tablet_manager.cc:1397] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 03:25:31.908653 14828 tablet_bootstrap.cc:492] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395: No bootstrap required, opened a new log
I20260809 03:25:31.908648 14827 tablet_bootstrap.cc:492] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: No bootstrap required, opened a new log
I20260809 03:25:31.908828 14827 ts_tablet_manager.cc:1397] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:25:31.908828 14828 ts_tablet_manager.cc:1397] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:25:31.911280 14826 raft_consensus.cc:348] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd99dc07bc6949ea922eeda4e57b0249" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 37921 } } peers { permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 } } peers { permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 } }
I20260809 03:25:31.911518 14826 raft_consensus.cc:374] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:31.911592 14826 raft_consensus.cc:729] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0289ff5c404c2282feb23a53423603, State: Initialized, Role: FOLLOWER
I20260809 03:25:31.912024 14826 consensus_queue.cc:260] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd99dc07bc6949ea922eeda4e57b0249" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 37921 } } peers { permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 } } peers { permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 } }
I20260809 03:25:31.912726 14671 heartbeater.cc:503] Master 127.9.46.254:34405 was elected leader, sending a full tablet report...
I20260809 03:25:31.913085 14826 ts_tablet_manager.cc:1428] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260809 03:25:31.913554 14827 raft_consensus.cc:348] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd99dc07bc6949ea922eeda4e57b0249" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 37921 } } peers { permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 } } peers { permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 } }
I20260809 03:25:31.913811 14827 raft_consensus.cc:374] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:31.913919 14827 raft_consensus.cc:729] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd99dc07bc6949ea922eeda4e57b0249, State: Initialized, Role: FOLLOWER
I20260809 03:25:31.913791 14828 raft_consensus.cc:348] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd99dc07bc6949ea922eeda4e57b0249" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 37921 } } peers { permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 } } peers { permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 } }
I20260809 03:25:31.914031 14828 raft_consensus.cc:374] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:31.914085 14828 raft_consensus.cc:729] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80029a5c5fed481498ff14bf4951f395, State: Initialized, Role: FOLLOWER
I20260809 03:25:31.914361 14827 consensus_queue.cc:260] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd99dc07bc6949ea922eeda4e57b0249" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 37921 } } peers { permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 } } peers { permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 } }
I20260809 03:25:31.914460 14828 consensus_queue.cc:260] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd99dc07bc6949ea922eeda4e57b0249" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 37921 } } peers { permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 } } peers { permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 } }
I20260809 03:25:31.915051 14809 heartbeater.cc:503] Master 127.9.46.254:34405 was elected leader, sending a full tablet report...
I20260809 03:25:31.915331 14827 ts_tablet_manager.cc:1428] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:25:31.915331 14828 ts_tablet_manager.cc:1428] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:25:31.915647 14532 heartbeater.cc:503] Master 127.9.46.254:34405 was elected leader, sending a full tablet report...
W20260809 03:25:31.917502 14672 tablet.cc:2369] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:31.964908 14833 raft_consensus.cc:482] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:31.965094 14833 raft_consensus.cc:504] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd99dc07bc6949ea922eeda4e57b0249" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 37921 } } peers { permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 } } peers { permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 } }
I20260809 03:25:31.966099 14833 leader_election.cc:288] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0289ff5c404c2282feb23a53423603 (127.9.46.194:41145), 80029a5c5fed481498ff14bf4951f395 (127.9.46.193:34569)
I20260809 03:25:31.968112 14811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.966393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:41145 (local address 127.9.46.195:37077)
0809 03:25:31.966514 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:31.966531 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:31.966624 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:31.967006 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:31.967009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:31.967026 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:31.967540 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:25:31.967542 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:31.967556 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:25:31.967588 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:31.967863 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:31.967867 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:31.967871 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:25:31.967947 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 03:25:31.967984 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 03:25:31.968112 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.966492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:37077 (local address 127.9.46.194:41145)
0809 03:25:31.966662 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.966667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.966687 (+ 20us) server_negotiation.cc:407] Connection header received
0809 03:25:31.966735 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.966738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.966781 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.966890 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.967719 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.967722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.967725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.967737 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.967758 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.967773 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.967867 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.967945 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.967986 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 03:25:31.968662 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.966659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:34569 (local address 127.9.46.195:39467)
0809 03:25:31.967118 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:31.967129 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:31.967239 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:31.967490 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:31.967492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:31.967508 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:31.967982 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:25:31.967985 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:31.967995 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:25:31.968029 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:31.968353 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:31.968356 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:31.968359 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 03:25:31.968434 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:25:31.968475 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":165,"threads_started":1}
I20260809 03:25:31.968667 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.966850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:39467 (local address 127.9.46.193:34569)
0809 03:25:31.966993 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.966998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.967217 (+ 219us) server_negotiation.cc:407] Connection header received
0809 03:25:31.967339 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.967346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.967379 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.967468 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.968160 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.968165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.968171 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.968185 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.968209 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.968223 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.968314 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.968447 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.968492 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:25:31.968801 14612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d6d81954b44b60be4b8c2bc25180e0" candidate_uuid: "dd99dc07bc6949ea922eeda4e57b0249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0289ff5c404c2282feb23a53423603" is_pre_election: true
I20260809 03:25:31.969128 14612 raft_consensus.cc:2454] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd99dc07bc6949ea922eeda4e57b0249 in term 0.
I20260809 03:25:31.969386 14484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d6d81954b44b60be4b8c2bc25180e0" candidate_uuid: "dd99dc07bc6949ea922eeda4e57b0249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80029a5c5fed481498ff14bf4951f395" is_pre_election: true
I20260809 03:25:31.969625 14693 leader_election.cc:302] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd0289ff5c404c2282feb23a53423603, dd99dc07bc6949ea922eeda4e57b0249; no voters:
I20260809 03:25:31.969707 14484 raft_consensus.cc:2454] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd99dc07bc6949ea922eeda4e57b0249 in term 0.
I20260809 03:25:31.969920 14833 raft_consensus.cc:2781] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:31.970057 14833 raft_consensus.cc:482] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:31.970124 14833 raft_consensus.cc:3037] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:31.971364 14833 raft_consensus.cc:504] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd99dc07bc6949ea922eeda4e57b0249" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 37921 } } peers { permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 } } peers { permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 } }
I20260809 03:25:31.971689 14833 leader_election.cc:288] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [CANDIDATE]: Term 1 election: Requested vote from peers cd0289ff5c404c2282feb23a53423603 (127.9.46.194:41145), 80029a5c5fed481498ff14bf4951f395 (127.9.46.193:34569)
I20260809 03:25:31.972051 14484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d6d81954b44b60be4b8c2bc25180e0" candidate_uuid: "dd99dc07bc6949ea922eeda4e57b0249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80029a5c5fed481498ff14bf4951f395"
I20260809 03:25:31.972084 14612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d6d81954b44b60be4b8c2bc25180e0" candidate_uuid: "dd99dc07bc6949ea922eeda4e57b0249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0289ff5c404c2282feb23a53423603"
I20260809 03:25:31.972188 14484 raft_consensus.cc:3037] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:31.972203 14612 raft_consensus.cc:3037] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:31.973341 14484 raft_consensus.cc:2454] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd99dc07bc6949ea922eeda4e57b0249 in term 1.
I20260809 03:25:31.973392 14612 raft_consensus.cc:2454] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd99dc07bc6949ea922eeda4e57b0249 in term 1.
I20260809 03:25:31.973665 14693 leader_election.cc:302] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd0289ff5c404c2282feb23a53423603, dd99dc07bc6949ea922eeda4e57b0249; no voters:
I20260809 03:25:31.973946 14833 raft_consensus.cc:2781] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:31.974187 14833 raft_consensus.cc:686] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 1 LEADER]: Becoming Leader. State: Replica: dd99dc07bc6949ea922eeda4e57b0249, State: Running, Role: LEADER
I20260809 03:25:31.974511 14833 consensus_queue.cc:237] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd99dc07bc6949ea922eeda4e57b0249" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 37921 } } peers { permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 } } peers { permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 } }
I20260809 03:25:31.977634 14337 catalog_manager.cc:5495] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd99dc07bc6949ea922eeda4e57b0249 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "dd99dc07bc6949ea922eeda4e57b0249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd99dc07bc6949ea922eeda4e57b0249" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 37921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 } health_report { overall_health: UNKNOWN } } }
W20260809 03:25:31.986949 14533 tablet.cc:2369] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:31.988696 14821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:31.987175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.9.46.195:37921)
0809 03:25:31.987284 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:31.987288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:31.987365 (+ 77us) server_negotiation.cc:407] Connection header received
0809 03:25:31.987498 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:31.987500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:31.987585 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:31.987673 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:31.988313 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:31.988316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:31.988318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:31.988328 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:31.988345 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:31.988353 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:31.988424 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:31.988483 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 03:25:31.988513 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:25:31.994365 14612 raft_consensus.cc:1261] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603 [term 1 FOLLOWER]: Refusing update from remote peer dd99dc07bc6949ea922eeda4e57b0249: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:31.994365 14484 raft_consensus.cc:1261] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395 [term 1 FOLLOWER]: Refusing update from remote peer dd99dc07bc6949ea922eeda4e57b0249: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:31.995043 14833 consensus_queue.cc:1035] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80029a5c5fed481498ff14bf4951f395" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:31.995275 14836 consensus_queue.cc:1035] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0289ff5c404c2282feb23a53423603" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 41145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:32.012818 14335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:32.012970 14335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:32.015377 14335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f47be19d720a4089a5f92b16b47c79e1: Sending DeleteTablet for 3 replicas of tablet 72d6d81954b44b60be4b8c2bc25180e0
I20260809 03:25:32.016099 14740 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d6d81954b44b60be4b8c2bc25180e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:32 UTC) from {username='slave'} at 127.0.0.1:51554
I20260809 03:25:32.016206 14592 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d6d81954b44b60be4b8c2bc25180e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:32 UTC) from {username='slave'} at 127.0.0.1:57214
I20260809 03:25:32.016283 14464 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d6d81954b44b60be4b8c2bc25180e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:32 UTC) from {username='slave'} at 127.0.0.1:42220
I20260809 03:25:32.016637 14852 tablet_replica.cc:330] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: stopping tablet replica
I20260809 03:25:32.016870 14851 tablet_replica.cc:330] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: stopping tablet replica
I20260809 03:25:32.017035 14852 raft_consensus.cc:2229] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:32.017112 14853 tablet_replica.cc:330] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395: stopping tablet replica
I20260809 03:25:32.017427 14852 raft_consensus.cc:2258] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:32.017459 14853 raft_consensus.cc:2229] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:32.017486 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 14401
I20260809 03:25:32.017815 14853 raft_consensus.cc:2258] T 72d6d81954b44b60be4b8c2bc25180e0 P 80029a5c5fed481498ff14bf4951f395 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:32.019731 14852 ts_tablet_manager.cc:1905] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:32.022644 14852 ts_tablet_manager.cc:1918] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:32.022806 14852 log.cc:1196] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/72d6d81954b44b60be4b8c2bc25180e0
I20260809 03:25:32.023213 14852 ts_tablet_manager.cc:1939] T 72d6d81954b44b60be4b8c2bc25180e0 P cd0289ff5c404c2282feb23a53423603: Deleting consensus metadata
I20260809 03:25:32.024520 14324 catalog_manager.cc:4841] TS cd0289ff5c404c2282feb23a53423603 (127.9.46.194:41145): tablet 72d6d81954b44b60be4b8c2bc25180e0 (table test-table [id=bd11f67b71974da0b0e88e42c20c3ad7]) successfully deleted
I20260809 03:25:32.025303 14851 raft_consensus.cc:2229] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:32.025775 14851 raft_consensus.cc:2258] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:32.027305 14851 ts_tablet_manager.cc:1905] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:32.029572 14851 ts_tablet_manager.cc:1918] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:32.029703 14851 log.cc:1196] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/72d6d81954b44b60be4b8c2bc25180e0
I20260809 03:25:32.030038 14851 ts_tablet_manager.cc:1939] T 72d6d81954b44b60be4b8c2bc25180e0 P dd99dc07bc6949ea922eeda4e57b0249: Deleting consensus metadata
I20260809 03:25:32.031126 14322 catalog_manager.cc:4841] TS dd99dc07bc6949ea922eeda4e57b0249 (127.9.46.195:37921): tablet 72d6d81954b44b60be4b8c2bc25180e0 (table test-table [id=bd11f67b71974da0b0e88e42c20c3ad7]) successfully deleted
W20260809 03:25:32.032156 14324 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:34569 (error 108)
I20260809 03:25:32.033066 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 14535
I20260809 03:25:32.033355 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:32.032983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:34569 (local address 127.0.0.1:42232)
0809 03:25:32.033162 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:32.033249 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 03:25:32.035627 14324 catalog_manager.cc:4584] TS 80029a5c5fed481498ff14bf4951f395 (127.9.46.193:34569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72d6d81954b44b60be4b8c2bc25180e0: Network error: Client connection negotiation failed: client connection to 127.9.46.193:34569: connect: Connection refused (error 111)
I20260809 03:25:32.042313 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 14674
I20260809 03:25:32.049336 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 14304
2026-08-09T03:25:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (983 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 03:25:32.072490 9403 mini_oidc.cc:152] Starting JWKS server
I20260809 03:25:32.074272 9403 webserver.cc:460] Webserver started at https://127.0.0.1:36005/ using document root <none> and password file <none>
I20260809 03:25:32.074468 9403 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 03:25:32.074764 9403 webserver.cc:460] Webserver started at http://127.0.0.1:41927/ using document root <none> and password file <none>
2026-08-09T03:25:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:32Z Disabled control of system clock
I20260809 03:25:32.085013 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:36093
--webserver_interface=127.9.46.254
--webserver_port=0
--builtin_ntp_servers=127.9.46.212:43241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:36093
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36005/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/testchainca.cert with env {}
W20260809 03:25:32.195393 14860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:32.195662 14860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:32.195703 14860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:32.199777 14860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:32.199848 14860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:32.199864 14860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:32.199882 14860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:32.199898 14860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:25:32.199914 14860 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:25:32.199927 14860 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36005/jwks.json
I20260809 03:25:32.205461 14860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:36093
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:36093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36005/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:32.206569 14860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:32.207551 14860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:32.216328 14865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:32.216310 14868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:32.216310 14866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:32.216727 14860 server_base.cc:1034] running on GCE node
I20260809 03:25:32.217340 14860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:32.218436 14860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:32.219631 14860 hybrid_clock.cc:648] HybridClock initialized: now 1786245932219613 us; error 30 us; skew 500 ppm
I20260809 03:25:32.221653 14860 webserver.cc:460] Webserver started at http://127.9.46.254:38549/ using document root <none> and password file <none>
I20260809 03:25:32.222265 14860 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:32.222380 14860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:32.222621 14860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:32.224749 14860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "cfc1cad214ba454bafeff8de644adcd8"
format_stamp: "Formatted at 2026-08-09 03:25:32 on dist-test-slave-65mx"
server_key: "b2e2e5ed9790aae0663f3b6d8cd2fbd7"
server_key_iv: "66cf545b3b0c493520d984f4cdadaaa2"
server_key_version: "encryptionkey@0"
I20260809 03:25:32.225334 14860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "cfc1cad214ba454bafeff8de644adcd8"
format_stamp: "Formatted at 2026-08-09 03:25:32 on dist-test-slave-65mx"
server_key: "b2e2e5ed9790aae0663f3b6d8cd2fbd7"
server_key_iv: "66cf545b3b0c493520d984f4cdadaaa2"
server_key_version: "encryptionkey@0"
I20260809 03:25:32.229031 14860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:25:32.231513 14874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:32.232818 14860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:32.232957 14860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "cfc1cad214ba454bafeff8de644adcd8"
format_stamp: "Formatted at 2026-08-09 03:25:32 on dist-test-slave-65mx"
server_key: "b2e2e5ed9790aae0663f3b6d8cd2fbd7"
server_key_iv: "66cf545b3b0c493520d984f4cdadaaa2"
server_key_version: "encryptionkey@0"
I20260809 03:25:32.233078 14860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:32.244500 14860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:32.245210 14860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:32.245409 14860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:32.252295 14860 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:36093
I20260809 03:25:32.252326 14929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:36093 every 8 connection(s)
I20260809 03:25:32.253222 14860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:32.255928 14930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:32.260723 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 14860
I20260809 03:25:32.260871 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:32.261297 9403 external_mini_cluster.cc:1428] Setting key 98c8cfc7bdba80ca4c151147a6f8d1fd
I20260809 03:25:32.261751 14930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8: Bootstrap starting.
I20260809 03:25:32.264426 14930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:32.265343 14930 log.cc:824] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:32.267489 14930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8: No bootstrap required, opened a new log
I20260809 03:25:32.270879 14933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:32.262802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.9.46.254:36093)
0809 03:25:32.263498 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:32.263512 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:32.263550 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:25:32.264279 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:32.264295 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:32.264769 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:32.265049 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:32.265398 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.266365 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.267538 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.268050 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.268174 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:32.269093 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:32.269098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:32.269110 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:32.269486 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:32.269541 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:32.269562 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:32.269646 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:32.269820 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 03:25:32.270150 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":162,"threads_started":1}
I20260809 03:25:32.271582 14930 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1cad214ba454bafeff8de644adcd8" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36093 } }
I20260809 03:25:32.271796 14930 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:32.271881 14930 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc1cad214ba454bafeff8de644adcd8, State: Initialized, Role: FOLLOWER
I20260809 03:25:32.272333 14930 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1cad214ba454bafeff8de644adcd8" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36093 } }
I20260809 03:25:32.272490 14930 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:32.272585 14930 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:32.272691 14930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:32.273844 14930 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1cad214ba454bafeff8de644adcd8" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36093 } }
I20260809 03:25:32.273977 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:25:32.274207 14930 leader_election.cc:302] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfc1cad214ba454bafeff8de644adcd8; no voters:
I20260809 03:25:32.274539 14930 leader_election.cc:288] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:32.274652 14936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:32.274915 14936 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [term 1 LEADER]: Becoming Leader. State: Replica: cfc1cad214ba454bafeff8de644adcd8, State: Running, Role: LEADER
I20260809 03:25:32.275269 14936 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1cad214ba454bafeff8de644adcd8" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36093 } }
I20260809 03:25:32.275822 14930 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:32.276443 14936 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfc1cad214ba454bafeff8de644adcd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1cad214ba454bafeff8de644adcd8" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36093 } } }
I20260809 03:25:32.276582 14936 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:32.276804 14936 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfc1cad214ba454bafeff8de644adcd8. Latest consensus state: current_term: 1 leader_uuid: "cfc1cad214ba454bafeff8de644adcd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc1cad214ba454bafeff8de644adcd8" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36093 } } }
I20260809 03:25:32.276901 14936 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:32.277024 14943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:32.280332 14943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:32.285574 14943 catalog_manager.cc:1347] Generated new cluster ID: 8cbbba1b234e43c7894ef06c33312be3
I20260809 03:25:32.285662 14943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:32.311666 14943 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:32.312723 14943 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:32.323243 14943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfc1cad214ba454bafeff8de644adcd8: Generated new TSK 0
I20260809 03:25:32.323951 14943 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 03:25:32.354966 14962 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 03:25:32.355116 14962 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 03:25:32.355767 14933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:32.337796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47924 (local address 127.9.46.254:36093)
0809 03:25:32.337954 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:32.337959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:32.338154 (+ 195us) server_negotiation.cc:407] Connection header received
0809 03:25:32.338349 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:32.338353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:32.338411 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:32.338523 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:25:32.338763 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.339624 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.340494 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.340884 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.340993 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:32.341305 (+ 312us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:25:32.355298 (+ 13993us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 03:25:32.355609 (+ 311us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47924: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 03:25:32.355854 14933 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47924: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 03:25:32.356986 14961 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:32.337719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:36093 (local address 127.0.0.1:47924)
0809 03:25:32.338065 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:32.338078 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:32.338192 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:32.338541 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:32.338543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:32.338550 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:25:32.338642 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:32.338647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.339759 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.339761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:32.340374 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:32.340381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.340997 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.341000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:32.341063 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:32.341105 (+ 42us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:25:32.355542 (+ 14437us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 03:25:32.355758 (+ 216us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.46.254:36093: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":111,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.46.254:36093: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36005/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 03:25:32.357781 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 14860
2026-08-09T03:25:32Z chronyd exiting
I20260809 03:25:32.397538 9403 test_util.cc:175] -----------------------------------------------
I20260809 03:25:32.397646 9403 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786245904130257-9403-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (326 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 03:25:32.398763 9403 mini_oidc.cc:152] Starting JWKS server
I20260809 03:25:32.399626 9403 webserver.cc:460] Webserver started at https://127.0.0.1:45587/ using document root <none> and password file <none>
I20260809 03:25:32.399763 9403 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 03:25:32.400064 9403 webserver.cc:460] Webserver started at http://127.0.0.1:34927/ using document root <none> and password file <none>
2026-08-09T03:25:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:32Z Disabled control of system clock
I20260809 03:25:32.409943 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41041
--webserver_interface=127.9.46.254
--webserver_port=0
--builtin_ntp_servers=127.9.46.212:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:41041
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45587/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/testchainca.cert with env {}
W20260809 03:25:32.519701 14970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:32.519958 14970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:32.520004 14970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:32.524052 14970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:32.524118 14970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:32.524134 14970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:32.524150 14970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:32.524165 14970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:25:32.524210 14970 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:25:32.524247 14970 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45587/jwks.json
I20260809 03:25:32.529781 14970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:41041
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45587/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:32.530853 14970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:32.531862 14970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:25:32.540447 14970 server_base.cc:1034] running on GCE node
W20260809 03:25:32.540393 14978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:32.540373 14976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:32.540395 14975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:32.541215 14970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:32.542320 14970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:32.543504 14970 hybrid_clock.cc:648] HybridClock initialized: now 1786245932543472 us; error 43 us; skew 500 ppm
I20260809 03:25:32.545441 14970 webserver.cc:460] Webserver started at http://127.9.46.254:35277/ using document root <none> and password file <none>
I20260809 03:25:32.546020 14970 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:32.546083 14970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:32.546337 14970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:32.548357 14970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "14bc177f205a46e8994f0019178667db"
format_stamp: "Formatted at 2026-08-09 03:25:32 on dist-test-slave-65mx"
server_key: "b8c7162dad06b60d45086292ffe25119"
server_key_iv: "3bf28f6c4c17c4dffcd25deddc89e217"
server_key_version: "encryptionkey@0"
I20260809 03:25:32.549007 14970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "14bc177f205a46e8994f0019178667db"
format_stamp: "Formatted at 2026-08-09 03:25:32 on dist-test-slave-65mx"
server_key: "b8c7162dad06b60d45086292ffe25119"
server_key_iv: "3bf28f6c4c17c4dffcd25deddc89e217"
server_key_version: "encryptionkey@0"
I20260809 03:25:32.552812 14970 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:25:32.555195 14984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:32.556363 14970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:32.556494 14970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "14bc177f205a46e8994f0019178667db"
format_stamp: "Formatted at 2026-08-09 03:25:32 on dist-test-slave-65mx"
server_key: "b8c7162dad06b60d45086292ffe25119"
server_key_iv: "3bf28f6c4c17c4dffcd25deddc89e217"
server_key_version: "encryptionkey@0"
I20260809 03:25:32.556667 14970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:32.588975 14970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:32.589973 14970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:32.590183 14970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:32.600751 14970 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:41041
I20260809 03:25:32.600777 15040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:41041 every 8 connection(s)
I20260809 03:25:32.602085 14970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:32.605953 15041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:32.607373 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 14970
I20260809 03:25:32.607503 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:32.607774 9403 external_mini_cluster.cc:1428] Setting key 92ed3c07872c9c276f2248b8d5c87b33
I20260809 03:25:32.614279 15041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db: Bootstrap starting.
I20260809 03:25:32.617594 15041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:32.618673 15041 log.cc:824] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:32.619239 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:32.609223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33080 (local address 127.9.46.254:41041)
0809 03:25:32.609945 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:32.609961 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:32.609999 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:25:32.610968 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:32.610990 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:32.611575 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:32.611987 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:32.612466 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.614076 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.615251 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.615975 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.616120 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:32.617021 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:32.617029 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:32.617044 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:32.617569 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:32.617663 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:32.617738 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:32.617833 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:32.617948 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 03:25:32.618220 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":151,"threads_started":1}
I20260809 03:25:32.621173 15041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db: No bootstrap required, opened a new log
I20260809 03:25:32.622427 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:25:32.627089 15041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bc177f205a46e8994f0019178667db" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41041 } }
I20260809 03:25:32.627409 15041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:32.627485 15041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14bc177f205a46e8994f0019178667db, State: Initialized, Role: FOLLOWER
I20260809 03:25:32.628007 15041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bc177f205a46e8994f0019178667db" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41041 } }
I20260809 03:25:32.628269 15041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:32.628368 15041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:32.628496 15041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:32.629556 15041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bc177f205a46e8994f0019178667db" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41041 } }
I20260809 03:25:32.629917 15041 leader_election.cc:302] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14bc177f205a46e8994f0019178667db; no voters:
I20260809 03:25:32.630234 15041 leader_election.cc:288] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:32.630384 15047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:32.630679 15047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [term 1 LEADER]: Becoming Leader. State: Replica: 14bc177f205a46e8994f0019178667db, State: Running, Role: LEADER
I20260809 03:25:32.631027 15047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bc177f205a46e8994f0019178667db" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41041 } }
I20260809 03:25:32.631539 15041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:32.632871 15048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14bc177f205a46e8994f0019178667db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bc177f205a46e8994f0019178667db" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41041 } } }
I20260809 03:25:32.632871 15049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14bc177f205a46e8994f0019178667db. Latest consensus state: current_term: 1 leader_uuid: "14bc177f205a46e8994f0019178667db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bc177f205a46e8994f0019178667db" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41041 } } }
I20260809 03:25:32.633054 15048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:32.633067 15049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:32.633523 15056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:32.636384 15056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:32.641637 15056 catalog_manager.cc:1347] Generated new cluster ID: 2b9895918e4f4f22830259e9719b145d
I20260809 03:25:32.641712 15056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:32.652621 15056 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:32.653833 15056 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:32.667351 15056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14bc177f205a46e8994f0019178667db: Generated new TSK 0
I20260809 03:25:32.668164 15056 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 03:25:32.708230 15073 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 03:25:32.708328 15073 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 03:25:32.709017 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:32.685587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.9.46.254:41041)
0809 03:25:32.685741 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:32.685748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:32.685922 (+ 174us) server_negotiation.cc:407] Connection header received
0809 03:25:32.686097 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:32.686099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:32.686145 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:32.686228 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:25:32.686497 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.687040 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.688309 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.688684 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.688768 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:32.689104 (+ 336us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:25:32.708651 (+ 19547us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 03:25:32.708895 (+ 244us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33084: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 03:25:32.709106 15044 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33084: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 03:25:32.709228 15072 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:32.685497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41041 (local address 127.0.0.1:33084)
0809 03:25:32.685826 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:32.685842 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:32.685961 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:32.686262 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:32.686265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:32.686273 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:25:32.686391 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:32.686398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.687167 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.687171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:32.688179 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:32.688193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.688809 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.688813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:32.688908 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:32.688937 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:25:32.708854 (+ 19917us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 03:25:32.709041 (+ 187us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.46.254:41041: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":116,"threads_started":1}
I20260809 03:25:32.710054 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 14970
2026-08-09T03:25:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (353 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 03:25:32.751873 9403 mini_oidc.cc:152] Starting JWKS server
I20260809 03:25:32.752856 9403 webserver.cc:460] Webserver started at https://127.0.0.1:35527/ using document root <none> and password file <none>
I20260809 03:25:32.752972 9403 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 03:25:32.753261 9403 webserver.cc:460] Webserver started at http://127.0.0.1:45465/ using document root <none> and password file <none>
2026-08-09T03:25:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:32Z Disabled control of system clock
I20260809 03:25:32.763511 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34809
--webserver_interface=127.9.46.254
--webserver_port=0
--builtin_ntp_servers=127.9.46.212:36119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:34809
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35527/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 03:25:32.900466 15081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:32.900835 15081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:32.900920 15081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:32.904978 15081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:32.905064 15081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:32.905094 15081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:32.905139 15081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:32.905177 15081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:25:32.905215 15081 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:25:32.905247 15081 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35527/jwks.json
I20260809 03:25:32.910143 15081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:36119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:34809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35527/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:32.911257 15081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:32.912709 15081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:32.921216 15089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:32.921188 15086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:32.921196 15087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:32.921900 15081 server_base.cc:1034] running on GCE node
I20260809 03:25:32.922610 15081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:32.924126 15081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:32.925339 15081 hybrid_clock.cc:648] HybridClock initialized: now 1786245932925306 us; error 38 us; skew 500 ppm
I20260809 03:25:32.927269 15081 webserver.cc:460] Webserver started at http://127.9.46.254:44385/ using document root <none> and password file <none>
I20260809 03:25:32.927812 15081 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:32.927873 15081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:32.928054 15081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:32.930174 15081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "51d7863fa71a46a28f9a90c10866fb5b"
format_stamp: "Formatted at 2026-08-09 03:25:32 on dist-test-slave-65mx"
server_key: "92a8fb1fd6062c7c2049e461f9602ba2"
server_key_iv: "c7f7523c72c1d78c372fc7f2340af3d1"
server_key_version: "encryptionkey@0"
I20260809 03:25:32.930702 15081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "51d7863fa71a46a28f9a90c10866fb5b"
format_stamp: "Formatted at 2026-08-09 03:25:32 on dist-test-slave-65mx"
server_key: "92a8fb1fd6062c7c2049e461f9602ba2"
server_key_iv: "c7f7523c72c1d78c372fc7f2340af3d1"
server_key_version: "encryptionkey@0"
I20260809 03:25:32.934340 15081 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 03:25:32.937131 15095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:32.938657 15081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:25:32.938789 15081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "51d7863fa71a46a28f9a90c10866fb5b"
format_stamp: "Formatted at 2026-08-09 03:25:32 on dist-test-slave-65mx"
server_key: "92a8fb1fd6062c7c2049e461f9602ba2"
server_key_iv: "c7f7523c72c1d78c372fc7f2340af3d1"
server_key_version: "encryptionkey@0"
I20260809 03:25:32.938938 15081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:32.952965 15081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:32.953698 15081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:32.953917 15081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:32.960949 15081 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:34809
I20260809 03:25:32.960969 15150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:34809 every 8 connection(s)
I20260809 03:25:32.961925 15081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:32.965288 15151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:32.969839 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 15081
I20260809 03:25:32.969973 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:32.970288 9403 external_mini_cluster.cc:1428] Setting key b882d135fc2c06560a63ce4bd34a0188
I20260809 03:25:32.971267 15151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b: Bootstrap starting.
I20260809 03:25:32.973666 15151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:32.974422 15151 log.cc:824] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:32.976293 15151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b: No bootstrap required, opened a new log
I20260809 03:25:32.979617 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:32.971733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50214 (local address 127.9.46.254:34809)
0809 03:25:32.972353 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:32.972368 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:32.972403 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:25:32.973095 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:32.973110 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:32.973490 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:32.973824 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:32.974270 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.975263 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.976169 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:32.976767 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:32.976941 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:32.977849 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:32.977853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:32.977866 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:32.978285 (+ 419us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:32.978339 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:32.978359 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:32.978471 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:32.978626 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 03:25:32.978886 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":130,"threads_started":1}
I20260809 03:25:32.980189 15151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d7863fa71a46a28f9a90c10866fb5b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34809 } }
I20260809 03:25:32.980427 15151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:32.980486 15151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51d7863fa71a46a28f9a90c10866fb5b, State: Initialized, Role: FOLLOWER
I20260809 03:25:32.980952 15151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d7863fa71a46a28f9a90c10866fb5b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34809 } }
I20260809 03:25:32.981094 15151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:32.981148 15151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:32.981230 15151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:32.982523 15151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d7863fa71a46a28f9a90c10866fb5b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34809 } }
I20260809 03:25:32.982741 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:25:32.982896 15151 leader_election.cc:302] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51d7863fa71a46a28f9a90c10866fb5b; no voters:
I20260809 03:25:32.983275 15151 leader_election.cc:288] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:32.983356 15156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:32.983551 15156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [term 1 LEADER]: Becoming Leader. State: Replica: 51d7863fa71a46a28f9a90c10866fb5b, State: Running, Role: LEADER
I20260809 03:25:32.983841 15156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d7863fa71a46a28f9a90c10866fb5b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34809 } }
I20260809 03:25:32.984611 15151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:32.985091 15156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51d7863fa71a46a28f9a90c10866fb5b. Latest consensus state: current_term: 1 leader_uuid: "51d7863fa71a46a28f9a90c10866fb5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d7863fa71a46a28f9a90c10866fb5b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34809 } } }
I20260809 03:25:32.985211 15156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:32.985520 15157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51d7863fa71a46a28f9a90c10866fb5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51d7863fa71a46a28f9a90c10866fb5b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34809 } } }
I20260809 03:25:32.985648 15160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:32.985662 15157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:32.989200 15160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:32.995298 15160 catalog_manager.cc:1347] Generated new cluster ID: d36f3238efaf427c8907e2b5bc51a5cb
I20260809 03:25:32.995381 15160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:33.024077 15160 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:33.025223 15160 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:33.033319 15160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51d7863fa71a46a28f9a90c10866fb5b: Generated new TSK 0
I20260809 03:25:33.034032 15160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:25:33.047719 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:33.042489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50216 (local address 127.9.46.254:34809)
0809 03:25:33.042629 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:33.042635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:33.042789 (+ 154us) server_negotiation.cc:407] Connection header received
0809 03:25:33.043012 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:33.043014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:33.043062 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:33.043174 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:25:33.043412 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:33.044099 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:33.045368 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:33.045584 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:33.045664 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:33.045986 (+ 322us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:25:33.047358 (+ 1372us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35527/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 03:25:33.047578 (+ 220us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50216: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35527/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 03:25:33.047791 15154 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50216: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35527/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 03:25:33.049664 15180 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:33.042397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34809 (local address 127.0.0.1:50216)
0809 03:25:33.042698 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:33.042713 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:33.042869 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:33.043182 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:33.043185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:33.043193 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:25:33.043305 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:33.043312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:33.044230 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:33.044233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:33.045238 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:33.045249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:33.045705 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:33.045709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:33.045799 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:33.045827 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:25:33.047531 (+ 1704us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35527/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 03:25:33.049459 (+ 1928us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.46.254:34809: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35527/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":1720,"thread_start_us":105,"threads_started":1}
I20260809 03:25:33.050026 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 15081
E20260809 03:25:33.062074 15181 webserver.cc:533] Webserver: sslize failed:
E20260809 03:25:33.062196 15181 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T03:25:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (342 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T03:25:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:33Z Disabled control of system clock
I20260809 03:25:33.108014 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43433
--webserver_interface=127.9.46.254
--webserver_port=0
--builtin_ntp_servers=127.9.46.212:33315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:43433
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/testchainca.cert with env {}
W20260809 03:25:33.218886 15185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:33.219161 15185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:33.219201 15185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:33.223269 15185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:33.223331 15185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:33.223347 15185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:33.223397 15185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:33.223429 15185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 03:25:33.223459 15185 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 03:25:33.223515 15185 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 03:25:33.229211 15185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:43433
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:33.230291 15185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:33.231297 15185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:33.239749 15191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:33.239749 15190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:33.239938 15185 server_base.cc:1034] running on GCE node
W20260809 03:25:33.239815 15193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:33.240685 15185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:33.241762 15185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:33.242923 15185 hybrid_clock.cc:648] HybridClock initialized: now 1786245933242897 us; error 41 us; skew 500 ppm
I20260809 03:25:33.244943 15185 webserver.cc:460] Webserver started at http://127.9.46.254:46621/ using document root <none> and password file <none>
I20260809 03:25:33.245530 15185 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:33.245594 15185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:33.245757 15185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:33.247740 15185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "4eddf64cfed646b1b41b3c88869e9494"
format_stamp: "Formatted at 2026-08-09 03:25:33 on dist-test-slave-65mx"
server_key: "678c9b41baa22115fdefb04a760bf939"
server_key_iv: "d84e9af9aafe4715add689d9fe86ba9a"
server_key_version: "encryptionkey@0"
I20260809 03:25:33.248252 15185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "4eddf64cfed646b1b41b3c88869e9494"
format_stamp: "Formatted at 2026-08-09 03:25:33 on dist-test-slave-65mx"
server_key: "678c9b41baa22115fdefb04a760bf939"
server_key_iv: "d84e9af9aafe4715add689d9fe86ba9a"
server_key_version: "encryptionkey@0"
I20260809 03:25:33.251911 15185 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 03:25:33.254462 15199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:33.255733 15185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 03:25:33.255851 15185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "4eddf64cfed646b1b41b3c88869e9494"
format_stamp: "Formatted at 2026-08-09 03:25:33 on dist-test-slave-65mx"
server_key: "678c9b41baa22115fdefb04a760bf939"
server_key_iv: "d84e9af9aafe4715add689d9fe86ba9a"
server_key_version: "encryptionkey@0"
I20260809 03:25:33.255944 15185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:33.277998 15185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:33.278693 15185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:33.278846 15185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:33.285948 15185 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:43433
I20260809 03:25:33.285969 15254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:43433 every 8 connection(s)
I20260809 03:25:33.286914 15185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:33.290202 15255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:33.294332 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 15185
I20260809 03:25:33.294468 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:33.294804 9403 external_mini_cluster.cc:1428] Setting key 4da6b16b90880b3fd7c59a605c21d313
I20260809 03:25:33.296373 15255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494: Bootstrap starting.
I20260809 03:25:33.298663 15255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:33.299412 15255 log.cc:824] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:33.301296 15255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494: No bootstrap required, opened a new log
I20260809 03:25:33.304013 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:33.296192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33480 (local address 127.9.46.254:43433)
0809 03:25:33.296903 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:33.296925 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:33.296977 (+ 52us) server_negotiation.cc:407] Connection header received
0809 03:25:33.297596 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:33.297612 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:33.297999 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:33.298330 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:33.298817 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:33.299862 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:33.300802 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:33.301273 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:33.301449 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:33.302339 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:33.302344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:33.302360 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:25:33.302735 (+ 375us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:33.302789 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:25:33.302809 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:33.302926 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:33.303083 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 03:25:33.303330 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":151,"threads_started":1}
I20260809 03:25:33.305154 15255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eddf64cfed646b1b41b3c88869e9494" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43433 } }
I20260809 03:25:33.305413 15255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:33.305483 15255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eddf64cfed646b1b41b3c88869e9494, State: Initialized, Role: FOLLOWER
I20260809 03:25:33.305938 15255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eddf64cfed646b1b41b3c88869e9494" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43433 } }
I20260809 03:25:33.306102 15255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:33.306185 15255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:33.306290 15255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:33.306974 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:25:33.307387 15255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eddf64cfed646b1b41b3c88869e9494" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43433 } }
I20260809 03:25:33.307714 15255 leader_election.cc:302] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4eddf64cfed646b1b41b3c88869e9494; no voters:
I20260809 03:25:33.308108 15255 leader_election.cc:288] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:33.308169 15260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:33.308387 15260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [term 1 LEADER]: Becoming Leader. State: Replica: 4eddf64cfed646b1b41b3c88869e9494, State: Running, Role: LEADER
I20260809 03:25:33.308699 15260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eddf64cfed646b1b41b3c88869e9494" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43433 } }
I20260809 03:25:33.309406 15255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:33.310420 15261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eddf64cfed646b1b41b3c88869e9494. Latest consensus state: current_term: 1 leader_uuid: "4eddf64cfed646b1b41b3c88869e9494" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eddf64cfed646b1b41b3c88869e9494" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43433 } } }
I20260809 03:25:33.310582 15261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:33.310827 15262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4eddf64cfed646b1b41b3c88869e9494" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eddf64cfed646b1b41b3c88869e9494" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43433 } } }
I20260809 03:25:33.310917 15262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:33.311079 15269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:33.313758 15269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:33.319028 15269 catalog_manager.cc:1347] Generated new cluster ID: d53fccc6ac584259bfbd96a5d9556e40
I20260809 03:25:33.319131 15269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:33.338145 15269 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:33.339455 15269 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:33.354117 15269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4eddf64cfed646b1b41b3c88869e9494: Generated new TSK 0
I20260809 03:25:33.354984 15269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:25:33.412388 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:33.367213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33484 (local address 127.9.46.254:43433)
0809 03:25:33.367340 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:33.367345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:33.367557 (+ 212us) server_negotiation.cc:407] Connection header received
0809 03:25:33.367722 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:33.367725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:33.367783 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:33.367901 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0809 03:25:33.368122 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:33.368992 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:33.369861 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:33.370213 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:33.370317 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:33.370612 (+ 295us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 03:25:33.411943 (+ 41331us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 03:25:33.412240 (+ 297us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33484: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 03:25:33.412475 15258 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33484: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 03:25:33.413043 15285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:33.367134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43433 (local address 127.0.0.1:33484)
0809 03:25:33.367469 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:33.367482 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:33.367583 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:33.367906 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:33.367908 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:33.367915 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0809 03:25:33.368018 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:33.368023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:33.369117 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:33.369120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:33.369748 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:33.369756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:33.370336 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:33.370338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:33.370412 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:33.370434 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 03:25:33.412297 (+ 41863us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 03:25:33.412717 (+ 420us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.46.254:43433: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":130,"threads_started":1}
I20260809 03:25:33.413805 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 15185
2026-08-09T03:25:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (341 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (123 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (127 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:33 dist-test-slave-65mx krb5kdc[15298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:33 dist-test-slave-65mx krb5kdc[15298](info): set up 2 sockets
Aug 09 03:25:33 dist-test-slave-65mx krb5kdc[15298](info): commencing operation
krb5kdc: starting...
W20260809 03:25:34.789371 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:34 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:34.919589 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43217
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:42701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:43217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:35.030862 15311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:35.031162 15311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:35.031262 15311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:35.035295 15311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:35.035429 15311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:35.035506 15311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:35.035570 15311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:35.035607 15311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:35.040977 15311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:42701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:43217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:35.042022 15311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:35.043044 15311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:35.051426 15317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:35.051440 15319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:35.051591 15311 server_base.cc:1034] running on GCE node
W20260809 03:25:35.051448 15316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:35.052520 15311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:35.053895 15311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:35.055092 15311 hybrid_clock.cc:648] HybridClock initialized: now 1786245935055074 us; error 36 us; skew 500 ppm
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.058099 15311 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:35.058964 15311 webserver.cc:460] Webserver started at http://127.9.46.254:33403/ using document root <none> and password file <none>
I20260809 03:25:35.059520 15311 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:35.059613 15311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:35.059844 15311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:35.061921 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "0555da9124734002a4ae9bf2c32f2771"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "7d62d7e7bc59a78c5e880927c20543f8"
server_key_iv: "c8880570a7aec0c220f2cebec487d192"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.062511 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "0555da9124734002a4ae9bf2c32f2771"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "7d62d7e7bc59a78c5e880927c20543f8"
server_key_iv: "c8880570a7aec0c220f2cebec487d192"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.066218 15311 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 03:25:35.068732 15327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:35.069941 15311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:25:35.070084 15311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "0555da9124734002a4ae9bf2c32f2771"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "7d62d7e7bc59a78c5e880927c20543f8"
server_key_iv: "c8880570a7aec0c220f2cebec487d192"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.070195 15311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:35.097298 15311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:35.100461 15311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:35.100701 15311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:35.107764 15311 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:43217
I20260809 03:25:35.107805 15381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:43217 every 8 connection(s)
I20260809 03:25:35.108728 15311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:35.111403 15382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:35.116761 15382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771: Bootstrap starting.
I20260809 03:25:35.117097 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 15311
I20260809 03:25:35.117241 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:35.117506 9403 external_mini_cluster.cc:1428] Setting key 5748fdcd96738da674a2230de82f69d2
I20260809 03:25:35.119659 15382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:35.120383 15382 log.cc:824] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:35.122453 15382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771: No bootstrap required, opened a new log
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:35.127009 15382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555da9124734002a4ae9bf2c32f2771" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43217 } }
I20260809 03:25:35.127296 15382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:35.127358 15382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0555da9124734002a4ae9bf2c32f2771, State: Initialized, Role: FOLLOWER
I20260809 03:25:35.127840 15382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555da9124734002a4ae9bf2c32f2771" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43217 } }
I20260809 03:25:35.127962 15382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:35.128062 15382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:35.128181 15382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:35.129132 15382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555da9124734002a4ae9bf2c32f2771" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43217 } }
I20260809 03:25:35.129518 15382 leader_election.cc:302] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0555da9124734002a4ae9bf2c32f2771; no voters:
I20260809 03:25:35.129887 15382 leader_election.cc:288] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:35.129990 15390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:35.130246 15390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [term 1 LEADER]: Becoming Leader. State: Replica: 0555da9124734002a4ae9bf2c32f2771, State: Running, Role: LEADER
I20260809 03:25:35.130570 15390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555da9124734002a4ae9bf2c32f2771" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43217 } }
I20260809 03:25:35.131129 15382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:35.132390 15391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0555da9124734002a4ae9bf2c32f2771" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555da9124734002a4ae9bf2c32f2771" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43217 } } }
I20260809 03:25:35.132530 15391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:35.132378 15392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0555da9124734002a4ae9bf2c32f2771. Latest consensus state: current_term: 1 leader_uuid: "0555da9124734002a4ae9bf2c32f2771" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0555da9124734002a4ae9bf2c32f2771" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43217 } } }
I20260809 03:25:35.132813 15392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:35.133006 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:35.118880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53020 (local address 127.9.46.254:43217)
0809 03:25:35.119297 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:35.119311 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:35.119345 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:25:35.120034 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:35.120051 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:35.120372 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:35.120761 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:35.121580 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.122538 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.123729 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.124201 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.124324 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:35.127185 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:35.127191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:35.127207 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:35.127238 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:35.129226 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.129795 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.129802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.129812 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.129960 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.130400 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.130404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.130406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.130927 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:35.131085 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:35.131604 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 03:25:35.131938 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":124,"threads_started":1}
W20260809 03:25:35.136729 15406 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:25:35.136809 15406 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:25:35.136870 15399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:35.137735 15399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:35.142952 15399 catalog_manager.cc:1347] Generated new cluster ID: bf297e3ca14d42ba946b0b8c2b32e312
I20260809 03:25:35.143024 15399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:35.157866 15399 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:35.159101 15399 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:35.172979 15399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0555da9124734002a4ae9bf2c32f2771: Generated new TSK 0
I20260809 03:25:35.173848 15399 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:25:35.203644 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43217
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:42701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:35.317157 15413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:35.317401 15413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:35.317442 15413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:35.321488 15413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:35.321547 15413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:35.321609 15413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:35.326661 15413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:42701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:35.327616 15413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:35.328533 15413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:35.338222 15418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:35.338222 15419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:35.338267 15421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:35.338495 15413 server_base.cc:1034] running on GCE node
I20260809 03:25:35.338864 15413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:35.339517 15413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:35.340704 15413 hybrid_clock.cc:648] HybridClock initialized: now 1786245935340691 us; error 24 us; skew 500 ppm
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.343357 15413 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:25:35.344178 15413 webserver.cc:460] Webserver started at http://127.9.46.193:39321/ using document root <none> and password file <none>
I20260809 03:25:35.344795 15413 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:35.344889 15413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:35.345124 15413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:35.347119 15413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "3c884ebcccc742fcbecc95cb591cf2ff"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "b54a997b428375295b0d57c78a75eb6a"
server_key_iv: "b968385043f51fdf1f4b122eb70a352f"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.347703 15413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "3c884ebcccc742fcbecc95cb591cf2ff"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "b54a997b428375295b0d57c78a75eb6a"
server_key_iv: "b968385043f51fdf1f4b122eb70a352f"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.351339 15413 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 03:25:35.353770 15428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:35.355002 15413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:25:35.355122 15413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "3c884ebcccc742fcbecc95cb591cf2ff"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "b54a997b428375295b0d57c78a75eb6a"
server_key_iv: "b968385043f51fdf1f4b122eb70a352f"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.355271 15413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:35.366977 15413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:35.369858 15413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:35.370009 15413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:35.370594 15413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:35.371395 15413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:35.371443 15413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:35.371526 15413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:35.371558 15413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:35.380972 15413 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:33907
I20260809 03:25:35.381171 15547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:33907 every 8 connection(s)
I20260809 03:25:35.381898 15413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:35.390596 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 15413
I20260809 03:25:35.390707 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:35.391036 9403 external_mini_cluster.cc:1428] Setting key 9f60b35168a95f0371277deda05fc140
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:35.396068 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:35.383833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:60309 (local address 127.9.46.254:43217)
0809 03:25:35.383963 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:35.383967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:35.384797 (+ 830us) server_negotiation.cc:407] Connection header received
0809 03:25:35.386254 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:35.386257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:35.386320 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:35.386398 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:35.387524 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.388064 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.389257 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.389515 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.389630 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:35.392375 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:35.392379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:35.392381 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:35.392415 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:35.393885 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.394411 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.394415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.394417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.394536 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.394945 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.394950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.394952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.395206 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:35.395317 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:35.395813 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 03:25:35.395948 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:25:35.396924 15550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:35.384143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43217 (local address 127.9.46.193:60309)
0809 03:25:35.384644 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:35.384688 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:35.386025 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:35.386559 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:35.386566 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:35.386907 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:35.387370 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:35.387383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.388204 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.388212 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:35.389098 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:35.389108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.389677 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.389681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:35.389866 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:35.390519 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:35.390535 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:35.392206 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:35.394017 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:35.394022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:35.394037 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:35.394302 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:35.394677 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:35.394683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:35.394688 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:35.394844 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:35.395317 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:35.395321 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:35.395568 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 03:25:35.395727 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 03:25:35.395977 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
I20260809 03:25:35.398159 15548 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43217
I20260809 03:25:35.398417 15548 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:35.399024 15548 heartbeater.cc:511] Master 127.9.46.254:43217 requested a full tablet report, sending...
I20260809 03:25:35.400797 15344 ts_manager.cc:194] Registered new tserver with Master: 3c884ebcccc742fcbecc95cb591cf2ff (127.9.46.193:33907)
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:25:35.402441 15344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:60309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:25:35.423326 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43217
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:42701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:35.534677 15553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:35.534915 15553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:35.534991 15553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:35.539085 15553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:35.539145 15553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:35.539249 15553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:35.544409 15553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:42701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:35.545425 15553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:35.546402 15553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:35.555727 15559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:35.555804 15562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:35.555799 15560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:35.556175 15553 server_base.cc:1034] running on GCE node
I20260809 03:25:35.556603 15553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:35.557250 15553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:35.558408 15553 hybrid_clock.cc:648] HybridClock initialized: now 1786245935558392 us; error 30 us; skew 500 ppm
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.564154 15553 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:25:35.565152 15553 webserver.cc:460] Webserver started at http://127.9.46.194:33315/ using document root <none> and password file <none>
I20260809 03:25:35.565713 15553 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:35.565809 15553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:35.566040 15553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:35.568118 15553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "9dca4964443f42b9aef4580c22c0d3cb"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "4c707611339369bd2f0f651131d31be8"
server_key_iv: "0fb44cbba1a0ea320a64ed4bad838a89"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.568864 15553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "9dca4964443f42b9aef4580c22c0d3cb"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "4c707611339369bd2f0f651131d31be8"
server_key_iv: "0fb44cbba1a0ea320a64ed4bad838a89"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.572533 15553 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:25:35.575086 15569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:35.576279 15553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:35.576414 15553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "9dca4964443f42b9aef4580c22c0d3cb"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "4c707611339369bd2f0f651131d31be8"
server_key_iv: "0fb44cbba1a0ea320a64ed4bad838a89"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.576525 15553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:35.615015 15553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:35.618091 15553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:35.618316 15553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:35.618942 15553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:35.619792 15553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:35.619863 15553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:35.619938 15553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:35.619982 15553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:35.629276 15553 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:38045
I20260809 03:25:35.629308 15687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:38045 every 8 connection(s)
I20260809 03:25:35.630172 15553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:35.630409 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 15553
I20260809 03:25:35.630535 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:35.630849 9403 external_mini_cluster.cc:1428] Setting key 665a5c3b19b9439705254f3b1bf931c2
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
I20260809 03:25:35.646291 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:35.632306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:39111 (local address 127.9.46.254:43217)
0809 03:25:35.632433 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:35.632438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:35.633578 (+ 1140us) server_negotiation.cc:407] Connection header received
0809 03:25:35.635763 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:35.635766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:35.635819 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:35.635909 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:35.637198 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.638107 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.639237 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.639635 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.639749 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:35.642550 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:35.642553 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:35.642555 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:35.642581 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:35.643937 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.644518 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.644521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.644522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.644683 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.645121 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.645124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.645126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.645324 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:35.645435 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:35.645967 (+ 532us) server_negotiation.cc:299] Negotiation successful
0809 03:25:35.646109 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:25:35.647104 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:35.632794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43217 (local address 127.9.46.194:39111)
0809 03:25:35.633411 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:35.633455 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:35.635543 (+ 2088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:35.636120 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:35.636126 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:35.636475 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:35.637038 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:35.637053 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.638232 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.638234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:35.639103 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:35.639112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.639803 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.639806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:35.639954 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:35.640649 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:35.640667 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:35.642375 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:35.644052 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:35.644057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:35.644071 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:35.644408 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:35.644780 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:35.644786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:35.644790 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:35.644991 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:35.645446 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:35.645455 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:35.645716 (+ 261us) client_negotiation.cc:769] Sending connection context
0809 03:25:35.645905 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 03:25:35.646244 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":143,"threads_started":1}
I20260809 03:25:35.648365 15688 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43217
I20260809 03:25:35.648676 15688 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:35.649195 15688 heartbeater.cc:511] Master 127.9.46.254:43217 requested a full tablet report, sending...
I20260809 03:25:35.650352 15344 ts_manager.cc:194] Registered new tserver with Master: 9dca4964443f42b9aef4580c22c0d3cb (127.9.46.194:38045)
I20260809 03:25:35.651263 15344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:39111
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:25:35.663833 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43217
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:42701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:35.777328 15693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:35.777591 15693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:35.777634 15693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:35.781713 15693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:35.781767 15693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:35.781824 15693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:35.786991 15693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:42701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:35.787990 15693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:35.788944 15693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:35.798403 15700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:35.798378 15698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:35.798393 15702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:35.798431 15693 server_base.cc:1034] running on GCE node
I20260809 03:25:35.798970 15693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:35.799647 15693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:35.800835 15693 hybrid_clock.cc:648] HybridClock initialized: now 1786245935800791 us; error 57 us; skew 500 ppm
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.803692 15693 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:25:35.804643 15693 webserver.cc:460] Webserver started at http://127.9.46.195:41875/ using document root <none> and password file <none>
I20260809 03:25:35.805168 15693 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:35.805228 15693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:35.805392 15693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:35.807456 15693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "f82518b64a37407496258249dc853648"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "7a65442ec3404940d35d3dc94dbf7b37"
server_key_iv: "59cd187d38416a624d1f478b4fab9def"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.808003 15693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "f82518b64a37407496258249dc853648"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "7a65442ec3404940d35d3dc94dbf7b37"
server_key_iv: "59cd187d38416a624d1f478b4fab9def"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.811906 15693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:25:35.814487 15710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:35.815711 15693 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:35.815840 15693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "f82518b64a37407496258249dc853648"
format_stamp: "Formatted at 2026-08-09 03:25:35 on dist-test-slave-65mx"
server_key: "7a65442ec3404940d35d3dc94dbf7b37"
server_key_iv: "59cd187d38416a624d1f478b4fab9def"
server_key_version: "encryptionkey@0"
I20260809 03:25:35.815963 15693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:35.826726 15693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:35.830044 15693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:35.830257 15693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:35.830967 15693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:35.831890 15693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:35.831938 15693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:35.832001 15693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:35.832036 15693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:35.841003 15693 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:33371
I20260809 03:25:35.841032 15829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:33371 every 8 connection(s)
I20260809 03:25:35.841868 15693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:35.850970 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 15693
I20260809 03:25:35.851097 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:35.851408 9403 external_mini_cluster.cc:1428] Setting key 504f6e04e96a636af97717e36795511d
I20260809 03:25:35.855512 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:35.843631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:44619 (local address 127.9.46.254:43217)
0809 03:25:35.843788 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:35.843793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:35.844698 (+ 905us) server_negotiation.cc:407] Connection header received
0809 03:25:35.846000 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:35.846003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:35.846050 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:35.846134 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:35.847231 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.847750 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.848925 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.849160 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.849242 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:35.851609 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:35.851613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:35.851616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:35.851645 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:35.853362 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.853991 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.853995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.853997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.854087 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.854440 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.854444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.854446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.854634 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:35.854741 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:35.855234 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 03:25:35.855349 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:25:35.856305 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:35.844056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43217 (local address 127.9.46.195:44619)
0809 03:25:35.844510 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:35.844569 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:35.845818 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:35.846298 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:35.846305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:35.846648 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:35.847095 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:35.847107 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.847855 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.847858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:35.848819 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:35.848829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.849294 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.849296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:35.849428 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:35.849937 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:35.849952 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:35.851425 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:35.853486 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:35.853491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:35.853505 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:35.853884 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:35.854189 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:35.854194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:35.854196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:35.854341 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:35.854755 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:35.854763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:35.854997 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 03:25:35.855190 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 03:25:35.855496 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":112,"threads_started":1}
I20260809 03:25:35.857498 15830 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43217
I20260809 03:25:35.857740 15830 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:35.858259 15830 heartbeater.cc:511] Master 127.9.46.254:43217 requested a full tablet report, sending...
I20260809 03:25:35.859288 15344 ts_manager.cc:194] Registered new tserver with Master: f82518b64a37407496258249dc853648 (127.9.46.195:33371)
I20260809 03:25:35.859858 15344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:44619
I20260809 03:25:35.865533 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/test-admin.keytab.
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.881618 9403 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 03:25:35.884639 15838 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:35.883077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42324 (local address 127.9.46.193:33907)
0809 03:25:35.883466 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:35.883485 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:35.883510 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:25:35.883563 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:35.883569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:35.883748 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:35.883880 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:35.884389 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42324: BlockingRecv error: recv got EOF from 127.0.0.1:42324 (error 108)
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":106,"threads_started":1}
W20260809 03:25:35.884694 9403 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5cc' of type FILE
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.885994 9403 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.887055 9403 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:25:35.897377 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:35.887998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.9.46.193:33907)
0809 03:25:35.888242 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:35.888246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:35.888379 (+ 133us) server_negotiation.cc:407] Connection header received
0809 03:25:35.888501 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:35.888504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:35.888588 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:35.888675 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:35.889205 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.890038 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.890849 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.891253 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.891345 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:35.893669 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:35.893679 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:35.893692 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:35.893717 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:35.895423 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.895934 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.895939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.895941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.896021 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.896267 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.896269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.896270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.896501 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:35.896723 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:35.897038 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 03:25:35.897224 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
W20260809 03:25:35.901170 15838 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:35.900175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.9.46.193:33907)
0809 03:25:35.900303 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:35.900308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:35.900424 (+ 116us) server_negotiation.cc:407] Connection header received
0809 03:25:35.900510 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:35.900512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:35.900597 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:35.900676 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:35.901012 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42338: BlockingRecv error: recv got EOF from 127.0.0.1:42338 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 03:25:35.901286 9403 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5cc' of type FILE
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.902298 9403 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.903154 9403 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:25:35.911888 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:35.903948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42346 (local address 127.9.46.193:33907)
0809 03:25:35.904092 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:35.904097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:35.904192 (+ 95us) server_negotiation.cc:407] Connection header received
0809 03:25:35.904339 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:35.904341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:35.904380 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:35.904470 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:35.904995 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.905503 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.906294 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.906532 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.906636 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:35.908997 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:35.909001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:35.909004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:35.909029 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:35.910261 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.910689 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.910692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.910694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.910779 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.911058 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.911061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.911069 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.911237 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:35.911367 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:35.911616 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 03:25:35.911727 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 03:25:35.914502 15838 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:35.913603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.9.46.193:33907)
0809 03:25:35.913743 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:35.913747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:35.913774 (+ 27us) server_negotiation.cc:407] Connection header received
0809 03:25:35.913857 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:35.913859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:35.913893 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:35.913948 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:35.914401 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42350: BlockingRecv error: recv got EOF from 127.0.0.1:42350 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 03:25:35.914577 9403 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786245904130257-9403-0/krb5kdc/krb5cc' of type FILE
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.915539 9403 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:35.916392 9403 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 03:25:35 dist-test-slave-65mx krb5kdc[15298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:25:35.924863 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:35.917241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.9.46.193:33907)
0809 03:25:35.917363 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:35.917367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:35.917449 (+ 82us) server_negotiation.cc:407] Connection header received
0809 03:25:35.917596 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:35.917598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:35.917632 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:35.917713 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:35.918201 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.918684 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.919459 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:35.919735 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:35.919844 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:35.922066 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:35.922069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:35.922072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:35.922100 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:35.923255 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.923670 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.923674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.923676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.923756 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:35.924032 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:35.924036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:35.924038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:35.924207 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:35.924337 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:35.924612 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 03:25:35.924723 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:25:35.925809 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 15413
I20260809 03:25:35.935940 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 15553
I20260809 03:25:35.945168 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 15693
I20260809 03:25:35.953054 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 15311
2026-08-09T03:25:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2290 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:36 dist-test-slave-65mx krb5kdc[15851](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:36 dist-test-slave-65mx krb5kdc[15851](info): set up 2 sockets
Aug 09 03:25:36 dist-test-slave-65mx krb5kdc[15851](info): commencing operation
krb5kdc: starting...
W20260809 03:25:37.041400 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:37 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.254.keytab.
Entry for principal oryx/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.254.keytab.
I20260809 03:25:37.172080 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43197
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.254.keytab
--principal=oryx/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:43197
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:37.284940 15865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:37.285259 15865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:37.285342 15865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:37.289307 15865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:37.289402 15865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:37.289443 15865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:37.289501 15865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:37.289538 15865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:37.294776 15865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:43197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.254.keytab
--principal=oryx/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:37.295909 15865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:37.297035 15865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:37.305469 15873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:37.305480 15871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:37.305537 15870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:37.305609 15865 server_base.cc:1034] running on GCE node
I20260809 03:25:37.306391 15865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:37.307466 15865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:37.308617 15865 hybrid_clock.cc:648] HybridClock initialized: now 1786245937308600 us; error 35 us; skew 500 ppm
Aug 09 03:25:37 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:37.311311 15865 init.cc:377] Logged in from keytab as oryx/127.9.46.254@KRBTEST.COM (short username oryx)
I20260809 03:25:37.312180 15865 webserver.cc:460] Webserver started at http://127.9.46.254:34527/ using document root <none> and password file <none>
I20260809 03:25:37.312757 15865 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:37.312822 15865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:37.312988 15865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:37.314961 15865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "6d8ac0b2b6324b258134eff073a6aa4f"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "10c7292372aee9dca2ce206f233a8c85"
server_key_iv: "dad57edd4910d0a72d058b167a5b6f9d"
server_key_version: "encryptionkey@0"
I20260809 03:25:37.315482 15865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "6d8ac0b2b6324b258134eff073a6aa4f"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "10c7292372aee9dca2ce206f233a8c85"
server_key_iv: "dad57edd4910d0a72d058b167a5b6f9d"
server_key_version: "encryptionkey@0"
I20260809 03:25:37.319181 15865 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 03:25:37.321534 15880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:37.322732 15865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:37.322837 15865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "6d8ac0b2b6324b258134eff073a6aa4f"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "10c7292372aee9dca2ce206f233a8c85"
server_key_iv: "dad57edd4910d0a72d058b167a5b6f9d"
server_key_version: "encryptionkey@0"
I20260809 03:25:37.322927 15865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:37.350152 15865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:37.353305 15865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:37.353464 15865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:37.360978 15865 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:43197
I20260809 03:25:37.361300 15935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:43197 every 8 connection(s)
I20260809 03:25:37.362035 15865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:37.365203 15936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:37.369488 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 15865
I20260809 03:25:37.369608 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:37.369901 9403 external_mini_cluster.cc:1428] Setting key 3aed03095884c3f688e40a450910a6af
I20260809 03:25:37.371753 15936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f: Bootstrap starting.
I20260809 03:25:37.374635 15936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:37.375442 15936 log.cc:824] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:37.377954 15936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f: No bootstrap required, opened a new log
Aug 09 03:25:37 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:37.382329 15936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } }
I20260809 03:25:37.382609 15936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:37.382675 15936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d8ac0b2b6324b258134eff073a6aa4f, State: Initialized, Role: FOLLOWER
I20260809 03:25:37.383217 15936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } }
I20260809 03:25:37.383404 15936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:37.383502 15936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:37.383613 15936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:37.384677 15936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } }
I20260809 03:25:37.385069 15936 leader_election.cc:302] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d8ac0b2b6324b258134eff073a6aa4f; no voters:
I20260809 03:25:37.385444 15936 leader_election.cc:288] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:37.385551 15941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:37.385771 15941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 1 LEADER]: Becoming Leader. State: Replica: 6d8ac0b2b6324b258134eff073a6aa4f, State: Running, Role: LEADER
I20260809 03:25:37.385927 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:37.371477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36370 (local address 127.9.46.254:43197)
0809 03:25:37.371911 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:37.371933 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:37.371997 (+ 64us) server_negotiation.cc:407] Connection header received
0809 03:25:37.372730 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:37.372749 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:37.373151 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:37.373469 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:37.374492 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:37.375601 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:37.376847 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:37.377362 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:37.377492 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:37.380631 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:37.380635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:37.380649 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:37.380676 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:37.382465 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:37.383030 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:37.383036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:37.383042 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:37.383206 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:37.383550 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:37.383556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:37.383559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:37.384021 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:37.384249 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:37.384600 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 03:25:37.384916 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":144,"threads_started":1}
I20260809 03:25:37.386148 15941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } }
I20260809 03:25:37.386797 15936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:37.388932 15942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } }
I20260809 03:25:37.389051 15942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:37.389477 15955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:37.392359 15955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:37.389489 15943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d8ac0b2b6324b258134eff073a6aa4f. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } }
I20260809 03:25:37.394124 15943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:37.398236 15955 catalog_manager.cc:1347] Generated new cluster ID: a64fda5661794ba1a078bff84af4bc4a
I20260809 03:25:37.398304 15955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:37.414019 15955 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:37.415467 15955 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:37.426088 15955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f: Generated new TSK 0
I20260809 03:25:37.427059 15955 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab.
Entry for principal oryx/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab.
I20260809 03:25:37.459190 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43197
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab
--principal=oryx/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:37.575016 15962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:37.575284 15962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:37.575328 15962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:37.579375 15962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:37.579445 15962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:37.579504 15962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:37.584986 15962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab
--principal=oryx/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:37.586135 15962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:37.587260 15962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:37.597175 15967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:37.597208 15968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:37.597379 15962 server_base.cc:1034] running on GCE node
W20260809 03:25:37.597566 15970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:37.597973 15962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:37.598640 15962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:37.599830 15962 hybrid_clock.cc:648] HybridClock initialized: now 1786245937599818 us; error 26 us; skew 500 ppm
Aug 09 03:25:37 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:37.602828 15962 init.cc:377] Logged in from keytab as oryx/127.9.46.193@KRBTEST.COM (short username oryx)
I20260809 03:25:37.603721 15962 webserver.cc:460] Webserver started at http://127.9.46.193:34793/ using document root <none> and password file <none>
I20260809 03:25:37.604301 15962 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:37.604401 15962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:37.604707 15962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:37.606886 15962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "ffd2a2de7b574340948f0136a92d236e"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "949640eae4143f95ab7b96ea97900a18"
server_key_iv: "c25e5f56d7a082fa0d3f57fc6fd729aa"
server_key_version: "encryptionkey@0"
I20260809 03:25:37.607481 15962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "ffd2a2de7b574340948f0136a92d236e"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "949640eae4143f95ab7b96ea97900a18"
server_key_iv: "c25e5f56d7a082fa0d3f57fc6fd729aa"
server_key_version: "encryptionkey@0"
I20260809 03:25:37.611328 15962 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 03:25:37.613921 15980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:37.615350 15962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:25:37.615509 15962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "ffd2a2de7b574340948f0136a92d236e"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "949640eae4143f95ab7b96ea97900a18"
server_key_iv: "c25e5f56d7a082fa0d3f57fc6fd729aa"
server_key_version: "encryptionkey@0"
I20260809 03:25:37.615636 15962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:37.628149 15962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:37.631541 15962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:37.631748 15962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:37.632350 15962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:37.633248 15962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:37.633324 15962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:37.633417 15962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:37.633468 15962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:37.643167 15962 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:34455
I20260809 03:25:37.643201 16097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:34455 every 8 connection(s)
I20260809 03:25:37.644083 15962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:37.646200 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 15962
I20260809 03:25:37.646345 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:37.646692 9403 external_mini_cluster.cc:1428] Setting key bebc6ac0ce3e15bf8151bcc0bdba2032
Aug 09 03:25:37 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:25:37.659821 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:37.645970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:33511 (local address 127.9.46.254:43197)
0809 03:25:37.646157 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:37.646161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:37.647116 (+ 955us) server_negotiation.cc:407] Connection header received
0809 03:25:37.648692 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:37.648695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:37.648741 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:37.648811 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:37.650459 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:37.651002 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:37.652812 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:37.653044 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:37.653130 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:37.655872 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:37.655878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:37.655882 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:37.655912 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:37.657654 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:37.658173 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:37.658176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:37.658178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:37.658266 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:37.658665 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:37.658671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:37.658676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:37.658899 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:37.658987 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:37.659518 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 03:25:37.659650 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 03:25:37.660827 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:37.646337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.193:33511)
0809 03:25:37.646914 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:37.647016 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:37.648443 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:37.649008 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:37.649019 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:37.649562 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:37.650257 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:37.650279 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:37.651143 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:37.651149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:37.652677 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:37.652691 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:37.653211 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:37.653217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:37.653497 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:37.654209 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:37.654225 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:37.655691 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:37.657777 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:37.657782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:37.657794 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:37.658061 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:37.658364 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:37.658366 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:37.658367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:37.658556 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:37.658996 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:37.659002 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:37.659243 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 03:25:37.659471 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 03:25:37.659821 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":107,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.194@KRBTEST.COM" created.
I20260809 03:25:37.662317 16098 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43197
I20260809 03:25:37.662580 16098 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:37.663097 16098 heartbeater.cc:511] Master 127.9.46.254:43197 requested a full tablet report, sending...
I20260809 03:25:37.664710 15897 ts_manager.cc:194] Registered new tserver with Master: ffd2a2de7b574340948f0136a92d236e (127.9.46.193:34455)
I20260809 03:25:37.666379 15897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:33511
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab.
Entry for principal oryx/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab.
I20260809 03:25:37.680083 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43197
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab
--principal=oryx/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:37.793766 16103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:37.794029 16103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:37.794075 16103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:37.798067 16103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:37.798121 16103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:37.798185 16103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:37.803336 16103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab
--principal=oryx/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:37.804409 16103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:37.805477 16103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:37.815379 16110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:37.815367 16109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:37.815362 16112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:37.815459 16103 server_base.cc:1034] running on GCE node
I20260809 03:25:37.816020 16103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:37.816749 16103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:37.817911 16103 hybrid_clock.cc:648] HybridClock initialized: now 1786245937817876 us; error 45 us; skew 500 ppm
Aug 09 03:25:37 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:37.820824 16103 init.cc:377] Logged in from keytab as oryx/127.9.46.194@KRBTEST.COM (short username oryx)
I20260809 03:25:37.821776 16103 webserver.cc:460] Webserver started at http://127.9.46.194:44273/ using document root <none> and password file <none>
I20260809 03:25:37.822366 16103 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:37.822428 16103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:37.822672 16103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:37.824760 16103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "41c925bbe2c34b3db10dca58dca3ec1b"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "862abe6af4b60112b2c3a13642f971bb"
server_key_iv: "5597b5dddf54f6b0c2032545cda632db"
server_key_version: "encryptionkey@0"
I20260809 03:25:37.825318 16103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "41c925bbe2c34b3db10dca58dca3ec1b"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "862abe6af4b60112b2c3a13642f971bb"
server_key_iv: "5597b5dddf54f6b0c2032545cda632db"
server_key_version: "encryptionkey@0"
I20260809 03:25:37.829164 16103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:25:37.831672 16120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:37.833076 16103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:37.833216 16103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "41c925bbe2c34b3db10dca58dca3ec1b"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "862abe6af4b60112b2c3a13642f971bb"
server_key_iv: "5597b5dddf54f6b0c2032545cda632db"
server_key_version: "encryptionkey@0"
I20260809 03:25:37.833338 16103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:37.846282 16103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:37.849364 16103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:37.849588 16103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:37.850243 16103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:37.851140 16103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:37.851191 16103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:37.851269 16103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:37.851311 16103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:37.861613 16103 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:43137
I20260809 03:25:37.861650 16237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:43137 every 8 connection(s)
I20260809 03:25:37.862586 16103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:37.867484 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 16103
I20260809 03:25:37.867590 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:37.867901 9403 external_mini_cluster.cc:1428] Setting key ac009440de9c2b3898e98b1c68d35b91
Aug 09 03:25:37 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:37.877080 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:37.864464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:52249 (local address 127.9.46.254:43197)
0809 03:25:37.864698 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:37.864702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:37.865541 (+ 839us) server_negotiation.cc:407] Connection header received
0809 03:25:37.866795 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:37.866798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:37.866847 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:37.866925 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:37.868269 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:37.868892 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:37.870069 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:37.870306 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:37.870417 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:37.873253 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:37.873257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:37.873259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:37.873284 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:37.874807 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:37.875345 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:37.875348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:37.875350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:37.875433 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:37.875789 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:37.875794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:37.875796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:37.875985 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:37.876109 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:37.876753 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 03:25:37.876886 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
WARNING: no policy specified for oryx/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:25:37.878101 16240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:37.864843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.194:52249)
0809 03:25:37.865390 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:37.865433 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:37.866604 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:37.867120 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:37.867127 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:37.867476 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:37.868112 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:37.868128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:37.869031 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:37.869034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:37.869926 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:37.869936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:37.870473 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:37.870477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:37.870667 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:37.871297 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:37.871314 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:37.873063 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:37.874941 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:37.874947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:37.874963 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:37.875238 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:37.875529 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:37.875533 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:37.875535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:37.875691 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:37.876109 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:37.876117 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:37.876429 (+ 312us) client_negotiation.cc:769] Sending connection context
0809 03:25:37.876709 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 03:25:37.877075 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":205,"threads_started":1}
I20260809 03:25:37.879393 16238 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43197
I20260809 03:25:37.879669 16238 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:37.880277 16238 heartbeater.cc:511] Master 127.9.46.254:43197 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.195@KRBTEST.COM" created.
I20260809 03:25:37.881605 15897 ts_manager.cc:194] Registered new tserver with Master: 41c925bbe2c34b3db10dca58dca3ec1b (127.9.46.194:43137)
I20260809 03:25:37.882455 15897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:52249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab.
Entry for principal oryx/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab.
I20260809 03:25:37.897795 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43197
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab
--principal=oryx/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:38.011062 16244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:38.011379 16244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:38.011463 16244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:38.015481 16244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:38.015584 16244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:38.015707 16244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:38.021059 16244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab
--principal=oryx/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:38.022115 16244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:38.023118 16244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:38.032836 16250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:38.032878 16249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:38.032900 16244 server_base.cc:1034] running on GCE node
W20260809 03:25:38.032847 16252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:38.033390 16244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:38.034041 16244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:38.035231 16244 hybrid_clock.cc:648] HybridClock initialized: now 1786245938035211 us; error 30 us; skew 500 ppm
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:38.038146 16244 init.cc:377] Logged in from keytab as oryx/127.9.46.195@KRBTEST.COM (short username oryx)
I20260809 03:25:38.039037 16244 webserver.cc:460] Webserver started at http://127.9.46.195:44009/ using document root <none> and password file <none>
I20260809 03:25:38.039606 16244 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:38.039700 16244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:38.039930 16244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:38.042009 16244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "dfde0bfae5c34f62914fb7a50e962a74"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "d0ed23d8a712fca73e17d928e5f769b6"
server_key_iv: "7c5966600dc3c6376e0061b31e93bebb"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.042603 16244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "dfde0bfae5c34f62914fb7a50e962a74"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "d0ed23d8a712fca73e17d928e5f769b6"
server_key_iv: "7c5966600dc3c6376e0061b31e93bebb"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.046281 16244 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:25:38.048638 16259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.049863 16244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:38.050005 16244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "dfde0bfae5c34f62914fb7a50e962a74"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "d0ed23d8a712fca73e17d928e5f769b6"
server_key_iv: "7c5966600dc3c6376e0061b31e93bebb"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.050134 16244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:38.063041 16244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:38.065914 16244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:38.066113 16244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:38.066682 16244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:38.067472 16244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:38.067553 16244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.067627 16244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:38.067667 16244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.076572 16244 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:35363
I20260809 03:25:38.076604 16378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:35363 every 8 connection(s)
I20260809 03:25:38.077497 16244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:38.084766 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 16244
I20260809 03:25:38.084918 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:38.085258 9403 external_mini_cluster.cc:1428] Setting key fac709f28d38d68d143df302cfdd439c
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:38.091193 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.079440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:52667 (local address 127.9.46.254:43197)
0809 03:25:38.079601 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.079606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.080391 (+ 785us) server_negotiation.cc:407] Connection header received
0809 03:25:38.081620 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.081623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.081671 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.081745 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.082789 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.083336 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.084470 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.084754 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.084834 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.087876 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.087879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.087881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.087904 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.089337 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.089832 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.089835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.089836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.089915 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.090275 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.090277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.090279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.090450 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.090528 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.090967 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.091073 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:25:38.092005 16381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.079805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.195:52667)
0809 03:25:38.080237 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.080280 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.081422 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.081865 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.081870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.082209 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.082666 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.082679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.083444 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.083446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.084342 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.084351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.084916 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.084920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.085118 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.085826 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.085845 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.087731 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.089447 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.089452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.089463 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.089715 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.090016 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.090045 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.090046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.090182 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.090546 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.090551 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.090767 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.090916 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.091181 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":103,"threads_started":1}
I20260809 03:25:38.093194 16379 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43197
I20260809 03:25:38.093401 16379 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:38.093832 16379 heartbeater.cc:511] Master 127.9.46.254:43197 requested a full tablet report, sending...
I20260809 03:25:38.094812 15897 ts_manager.cc:194] Registered new tserver with Master: dfde0bfae5c34f62914fb7a50e962a74 (127.9.46.195:35363)
I20260809 03:25:38.095408 15897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:52667
I20260809 03:25:38.099347 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.46.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.253.keytab.
Entry for principal oryx/127.9.46.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.253.keytab.
I20260809 03:25:38.129738 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.253:45835
--webserver_interface=127.9.46.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.253.keytab
--principal=oryx/127.9.46.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:43197,127.9.46.253:45835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:38.241076 16384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:38.241331 16384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:38.241403 16384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:38.245414 16384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:38.245473 16384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:38.245492 16384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:38.245533 16384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:38.245568 16384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:38.250756 16384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:43197,127.9.46.253:45835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.253.keytab
--principal=oryx/127.9.46.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.253:45835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.46.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
W20260809 03:25:38.250990 16384 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.46.254:43197,127.9.46.253:45835'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 03:25:38.251871 16384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:38.252996 16384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:38.261438 16389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:38.261436 16392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:38.261446 16390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:38.261858 16384 server_base.cc:1034] running on GCE node
I20260809 03:25:38.262403 16384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:38.263496 16384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:38.264631 16384 hybrid_clock.cc:648] HybridClock initialized: now 1786245938264612 us; error 30 us; skew 500 ppm
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:38.267436 16384 init.cc:377] Logged in from keytab as oryx/127.9.46.253@KRBTEST.COM (short username oryx)
I20260809 03:25:38.268353 16384 webserver.cc:460] Webserver started at http://127.9.46.253:39041/ using document root <none> and password file <none>
I20260809 03:25:38.268957 16384 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:38.269093 16384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:38.269331 16384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:38.271366 16384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data/instance:
uuid: "e00171ca5d8d4df38c465537e5906d59"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "9599dbe2c27e0d2e5adbf9328ffd9dea"
server_key_iv: "b6c594f37ff835c9442c4fe9480b4531"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.271939 16384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal/instance:
uuid: "e00171ca5d8d4df38c465537e5906d59"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "9599dbe2c27e0d2e5adbf9328ffd9dea"
server_key_iv: "b6c594f37ff835c9442c4fe9480b4531"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.275825 16384 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:25:38.278438 16400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.279744 16384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 03:25:38.279892 16384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal
uuid: "e00171ca5d8d4df38c465537e5906d59"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "9599dbe2c27e0d2e5adbf9328ffd9dea"
server_key_iv: "b6c594f37ff835c9442c4fe9480b4531"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.280062 16384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:38.297078 16384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:38.299949 16384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:38.300187 16384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:38.307472 16384 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.253:45835
I20260809 03:25:38.307749 16455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.253:45835 every 8 connection(s)
I20260809 03:25:38.308470 16384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
I20260809 03:25:38.311450 16456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:38.315659 16456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:25:38.316480 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 16384
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.253@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.46.253@KRBTEST.COM
I20260809 03:25:38.332845 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.317972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.9.46.254:43197)
0809 03:25:38.318144 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.318151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.318945 (+ 794us) server_negotiation.cc:407] Connection header received
0809 03:25:38.319960 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.319963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.320027 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.320130 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.321413 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.322239 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.323981 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.324332 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.324471 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.327848 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.327852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.327854 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.327882 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.329897 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.330547 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.330551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.330553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.330636 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.331102 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.331106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.331107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.331383 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.331467 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.332224 (+ 757us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.332464 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 03:25:38.333820 16459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.318306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36386)
0809 03:25:38.318770 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.318815 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.319757 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.320213 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.320219 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.320770 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.321255 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.321278 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.322409 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.322412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.323828 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.323852 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.324553 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.324587 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.324756 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.325502 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.325523 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.327645 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.330041 (+ 2396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.330054 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.330065 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.330419 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.330756 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.330759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.330762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.330934 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.331498 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.331513 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.331821 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.332062 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.332451 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":61,"spinlock_wait_cycles":7040,"thread_start_us":119,"threads_started":1}
I20260809 03:25:38.333820 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.319087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49810 (local address 127.9.46.253:45835)
0809 03:25:38.319358 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.319366 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.319397 (+ 31us) server_negotiation.cc:407] Connection header received
0809 03:25:38.319770 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.319791 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.320012 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.320261 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.321265 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.322525 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.323852 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.324412 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.324630 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.328457 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.328464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.328479 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.328506 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.330180 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.330766 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.330770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.330771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.330917 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.331268 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.331273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.331275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.331821 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.332060 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.332826 (+ 766us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.332983 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":163,"spinlock_wait_cycles":3200,"thread_start_us":92,"threads_started":1}
I20260809 03:25:38.336112 16456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.253@KRBTEST.COM for oryx/127.9.46.253@KRBTEST.COM
I20260809 03:25:38.344626 16459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.336749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:45835 (local address 127.0.0.1:49814)
0809 03:25:38.336855 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.336867 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.336930 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.337298 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.337301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.337555 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.337680 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.337686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.338673 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.338676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.339184 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.339192 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.339681 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.339684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.339776 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.340285 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.340296 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.341655 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.343212 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.343214 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.343216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.343407 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.343660 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.343662 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.343663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.343738 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.344118 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.344120 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.344195 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.344275 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.344375 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 03:25:38.344635 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.336879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49814 (local address 127.9.46.253:45835)
0809 03:25:38.337022 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.337027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.337041 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:25:38.337093 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.337099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.337149 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.337239 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.337809 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.338493 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.339312 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.339570 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.339664 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.341808 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.341811 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.341813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.341834 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.343111 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.343503 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.343506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.343507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.343578 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.343840 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.343843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.343844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.344032 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.344116 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.344336 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.344429 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:25:38.347833 16456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Bootstrap starting.
I20260809 03:25:38.350176 16456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:38.350903 16456 log.cc:824] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:38.352914 16456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: No bootstrap required, opened a new log
I20260809 03:25:38.355911 16456 raft_consensus.cc:348] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } }
I20260809 03:25:38.356132 16456 raft_consensus.cc:374] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:38.356215 16456 raft_consensus.cc:729] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00171ca5d8d4df38c465537e5906d59, State: Initialized, Role: FOLLOWER
I20260809 03:25:38.356717 16456 consensus_queue.cc:260] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } }
I20260809 03:25:38.357213 16462 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } } }
I20260809 03:25:38.357326 16462 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:25:38.357901 16456 sys_catalog.cc:564] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 03:25:38.362672 16474 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:25:38.362758 16474 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:25:38.366004 16384 master_runner.cc:428] Detected that this master e00171ca5d8d4df38c465537e5906d59 is joining an existing cluster
I20260809 03:25:38.366073 16384 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.46.253:45835
I20260809 03:25:38.378084 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.368734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.9.46.253:45835)
0809 03:25:38.368838 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.368843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.369101 (+ 258us) server_negotiation.cc:407] Connection header received
0809 03:25:38.369244 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.369247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.369282 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.369326 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.370087 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.370868 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.372099 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.372421 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.372492 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.374530 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.374534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.374537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.374557 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.376261 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.376788 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.376792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.376793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.376865 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.377148 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.377154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.377155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.377368 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.377437 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.377778 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.377889 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 03:25:38.378158 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.368286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.9.46.254:43197)
0809 03:25:38.368409 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.368414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.369023 (+ 609us) server_negotiation.cc:407] Connection header received
0809 03:25:38.369199 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.369202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.369248 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.369301 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.370068 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.370871 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.372141 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.372464 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.372533 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.374254 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.374258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.374261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.374286 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.376325 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.376866 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.376869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.376871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.376968 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.377267 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.377271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.377272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.377463 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.377551 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.377889 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.378008 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:25:38.378212 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.368215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36396)
0809 03:25:38.368938 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.368952 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.369099 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.369478 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.369481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.369779 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.369921 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.369928 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.371020 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.371023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.372001 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.372010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.372717 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.372720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.372813 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.373584 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.373598 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.374102 (+ 504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.376456 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.376459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.376461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.376759 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.377081 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.377083 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.377085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.377167 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.377618 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.377621 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.377714 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.377814 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.377960 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260809 03:25:38.379144 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.368702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:45835 (local address 127.0.0.1:49824)
0809 03:25:38.369011 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.369034 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.369155 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.369478 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.369481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.369820 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.369956 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.369963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.370996 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.371000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.371968 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.371979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.372674 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.372677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.372769 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.373833 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.373848 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.374396 (+ 548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.376372 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.376375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.376377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.376690 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.376971 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.376974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.376976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.377060 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.377534 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.377536 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.377629 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.377995 (+ 366us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.378958 (+ 963us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":33,"spinlock_wait_cycles":8576,"thread_start_us":81,"threads_started":1}
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.254@KRBTEST.COM for oryx/127.9.46.253@KRBTEST.COM
I20260809 03:25:38.392874 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.382991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.9.46.253:45835)
0809 03:25:38.383116 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.383120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.383450 (+ 330us) server_negotiation.cc:407] Connection header received
0809 03:25:38.383639 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.383642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.383696 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.383761 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.384600 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.385127 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.386337 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.386637 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.386752 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.389179 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.389182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.389184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.389205 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.390945 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.391577 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.391580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.391584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.391673 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.391937 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.391940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.391942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.392167 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.392267 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.392619 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.392732 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:25:38.392915 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.382927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:45835 (local address 127.0.0.1:49826)
0809 03:25:38.383307 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.383361 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.383497 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.383822 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.383828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.384286 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.384417 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.384425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.385233 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.385237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.386215 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.386226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.386757 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.386761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.386879 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.387473 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.387495 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.389025 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.391107 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.391116 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.391124 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.391476 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.391766 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.391769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.391771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.391850 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.392271 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.392276 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.392387 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.392511 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.392786 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":118,"threads_started":1}
I20260809 03:25:38.394429 15897 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.46.253:45835: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e00171ca5d8d4df38c465537e5906d59"
member_type: NON_VOTER
last_known_addr {
host: "127.9.46.253"
port: 45835
}
attrs {
promote: true
}
}
dest_uuid: "6d8ac0b2b6324b258134eff073a6aa4f"
cas_config_opid_index: -1
I20260809 03:25:38.395150 15897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } }
I20260809 03:25:38.396865 16484 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e00171ca5d8d4df38c465537e5906d59 (127.9.46.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } } }
I20260809 03:25:38.397861 16427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:38.398427 16484 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.46.253:45835
I20260809 03:25:38.398855 16384 master.cc:537] Master@127.9.46.253:45835 shutting down...
I20260809 03:25:38.398910 16485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } } }
I20260809 03:25:38.399000 16485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: This master's current role is: LEADER
WARNING: no policy specified for oryx/127.9.46.252@KRBTEST.COM; defaulting to no policy
I20260809 03:25:38.399514 16427 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Refusing update from remote peer 6d8ac0b2b6324b258134eff073a6aa4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 03:25:38.400322 16485 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.252@KRBTEST.COM" created.
W20260809 03:25:38.402794 15883 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 03:25:38.403676 15883 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f -> Peer e00171ca5d8d4df38c465537e5906d59 (127.9.46.253:45835): Couldn't send request to peer e00171ca5d8d4df38c465537e5906d59. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 03:25:38.407359 16384 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:38.407688 16384 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:38.407814 16384 tablet_replica.cc:330] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: stopping tablet replica
I20260809 03:25:38.412117 16384 master.cc:552] Master@127.9.46.253:45835 shutdown complete.
I20260809 03:25:38.412168 16384 master_runner.cc:305] Clearing existing system tablet
I20260809 03:25:38.412421 16384 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:38.412497 16384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:38.413679 16384 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:38.414827 16490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.415079 16384 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:38.415223 16384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:38.415264 16384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal
uuid: "e00171ca5d8d4df38c465537e5906d59"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "9599dbe2c27e0d2e5adbf9328ffd9dea"
server_key_iv: "b6c594f37ff835c9442c4fe9480b4531"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.417905 16384 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.252.keytab.
Entry for principal oryx/127.9.46.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.252.keytab.
I20260809 03:25:38.420785 16384 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 03:25:38.420890 16384 log.cc:1196] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 03:25:38.421221 16384 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Deleting consensus metadata
I20260809 03:25:38.421413 16384 master_runner.cc:315] Copying system tablet from 127.9.46.254:43197
I20260809 03:25:38.421597 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.252:45927
--webserver_interface=127.9.46.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.252.keytab
--principal=oryx/127.9.46.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:43197,127.9.46.253:45835,127.9.46.252:45927
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:25:38.422847 16384 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: tablet copy: Beginning tablet copy session from remote peer at address 127.9.46.254:43197
I20260809 03:25:38.435738 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.423560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36408)
0809 03:25:38.426352 (+ 2792us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.426369 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.426514 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.426871 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.426874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.427103 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.427302 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.427311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.428506 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.428510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.429472 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.429482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.430233 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.430237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.430340 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.431126 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.431141 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.431594 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.433933 (+ 2339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.433937 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.433939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.434213 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.434553 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.434556 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.434558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.434653 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.435152 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.435155 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.435261 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.435370 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.435519 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2711,"thread_start_us":91,"threads_started":1}
I20260809 03:25:38.435740 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.423702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.9.46.254:43197)
0809 03:25:38.423850 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.423855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.426479 (+ 2624us) server_negotiation.cc:407] Connection header received
0809 03:25:38.426638 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.426644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.426739 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.426839 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.427440 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.428370 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.429620 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.429967 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.430053 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.431765 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.431772 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.431777 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.431815 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.433804 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.434325 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.434329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.434333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.434438 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.434761 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.434767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.434770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.435031 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.435146 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.435443 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.435572 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:25:38.436882 15918 tablet_copy_service.cc:140] P 6d8ac0b2b6324b258134eff073a6aa4f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e00171ca5d8d4df38c465537e5906d59 ({username='oryx', principal='oryx/127.9.46.253@KRBTEST.COM'} at 127.0.0.1:36408)
I20260809 03:25:38.437155 15918 tablet_copy_service.cc:161] P 6d8ac0b2b6324b258134eff073a6aa4f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e00171ca5d8d4df38c465537e5906d59 at {username='oryx', principal='oryx/127.9.46.253@KRBTEST.COM'} at 127.0.0.1:36408: session id = e00171ca5d8d4df38c465537e5906d59-00000000000000000000000000000000
I20260809 03:25:38.439157 15918 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 03:25:38.441149 16384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 03:25:38.443631 16384 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: tablet copy: Starting download of 0 data blocks...
I20260809 03:25:38.443845 16384 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: tablet copy: Starting download of 1 WAL segments...
I20260809 03:25:38.445706 16384 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 03:25:38.447553 15918 tablet_copy_service.cc:342] P 6d8ac0b2b6324b258134eff073a6aa4f: Request end of tablet copy session e00171ca5d8d4df38c465537e5906d59-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.46.253@KRBTEST.COM'} at 127.0.0.1:36408
I20260809 03:25:38.447679 15918 tablet_copy_service.cc:433] P 6d8ac0b2b6324b258134eff073a6aa4f: ending tablet copy session e00171ca5d8d4df38c465537e5906d59-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e00171ca5d8d4df38c465537e5906d59
I20260809 03:25:38.455581 16384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:38.457628 16500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:38.458019 16499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:38.458244 16502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:38.458806 16384 server_base.cc:1034] running on GCE node
I20260809 03:25:38.458971 16384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:38.459250 16384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:38.460398 16384 hybrid_clock.cc:648] HybridClock initialized: now 1786245938460386 us; error 30 us; skew 500 ppm
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:38.462040 16384 init.cc:377] Logged in from keytab as oryx/127.9.46.253@KRBTEST.COM (short username oryx)
I20260809 03:25:38.462477 16384 webserver.cc:460] Webserver started at http://127.9.46.253:37313/ using document root <none> and password file <none>
I20260809 03:25:38.462663 16384 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:38.462707 16384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:38.463975 16384 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:38.465008 16510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.465296 16384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:25:38.465371 16384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal
uuid: "e00171ca5d8d4df38c465537e5906d59"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "9599dbe2c27e0d2e5adbf9328ffd9dea"
server_key_iv: "b6c594f37ff835c9442c4fe9480b4531"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.465453 16384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:38.477509 16384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:38.479503 16384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:38.484390 16384 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.253:45835
I20260809 03:25:38.484503 16568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.253:45835 every 8 connection(s)
I20260809 03:25:38.484993 16384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
I20260809 03:25:38.486879 16569 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:25:38.487982 16569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Bootstrap starting.
I20260809 03:25:38.498046 16569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:25:38.498382 16569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Bootstrap complete.
I20260809 03:25:38.498948 16569 raft_consensus.cc:348] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } }
I20260809 03:25:38.499099 16569 raft_consensus.cc:729] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e00171ca5d8d4df38c465537e5906d59, State: Initialized, Role: LEARNER
I20260809 03:25:38.499212 16569 consensus_queue.cc:260] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } }
I20260809 03:25:38.499558 16572 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } } }
I20260809 03:25:38.499660 16569 sys_catalog.cc:564] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:38.499770 16572 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: This master's current role is: LEARNER
I20260809 03:25:38.502974 16584 catalog_manager.cc:1259] Loaded cluster ID: a64fda5661794ba1a078bff84af4bc4a
I20260809 03:25:38.503119 16584 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: loading cluster ID for follower catalog manager: success
I20260809 03:25:38.505381 16584 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: acquiring CA information for follower catalog manager: success
I20260809 03:25:38.506807 16584 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 03:25:38.557427 16497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:38.557734 16497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:38.557823 16497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:38.561892 16497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:38.561972 16497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:38.562011 16497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:38.562070 16497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:38.562108 16497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:38.567716 16497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:43197,127.9.46.253:45835,127.9.46.252:45927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.252.keytab
--principal=oryx/127.9.46.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.252:45927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.46.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:38.568847 16497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:38.569861 16497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:38.578219 16589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:38.578205 16592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:38.578205 16590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:38.578650 16497 server_base.cc:1034] running on GCE node
I20260809 03:25:38.579262 16497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:38.580412 16497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:38.581573 16497 hybrid_clock.cc:648] HybridClock initialized: now 1786245938581559 us; error 26 us; skew 500 ppm
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:38.584358 16497 init.cc:377] Logged in from keytab as oryx/127.9.46.252@KRBTEST.COM (short username oryx)
I20260809 03:25:38.585283 16497 webserver.cc:460] Webserver started at http://127.9.46.252:37885/ using document root <none> and password file <none>
I20260809 03:25:38.585843 16497 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:38.585904 16497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:38.586155 16497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:38.588438 16497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data/instance:
uuid: "32a099fbf430442e8f90bbaf5dc69d92"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "1e5a9e4faaf5de3cd1542e9281df4032"
server_key_iv: "d45d7ab33b662bc3082c43530afcc0fd"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.589107 16497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal/instance:
uuid: "32a099fbf430442e8f90bbaf5dc69d92"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "1e5a9e4faaf5de3cd1542e9281df4032"
server_key_iv: "d45d7ab33b662bc3082c43530afcc0fd"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.592789 16497 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:25:38.595112 16599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.596318 16497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:38.596457 16497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal
uuid: "32a099fbf430442e8f90bbaf5dc69d92"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "1e5a9e4faaf5de3cd1542e9281df4032"
server_key_iv: "d45d7ab33b662bc3082c43530afcc0fd"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.596622 16497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:38.609993 16497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:38.612948 16497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:38.613160 16497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:38.620677 16497 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.252:45927
I20260809 03:25:38.620961 16654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.252:45927 every 8 connection(s)
I20260809 03:25:38.621773 16497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
I20260809 03:25:38.624651 16655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:38.628465 16655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:25:38.630072 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 16497
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.252@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.46.252@KRBTEST.COM
I20260809 03:25:38.643632 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.630563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36420 (local address 127.9.46.254:43197)
0809 03:25:38.630711 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.630715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.631649 (+ 934us) server_negotiation.cc:407] Connection header received
0809 03:25:38.632742 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.632746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.632796 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.632895 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.634040 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.634932 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.636271 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.636559 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.636659 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.639792 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.639796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.639801 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.639836 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.641371 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.641982 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.641986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.641988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.642098 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.642453 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.642457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.642459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.642652 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.642760 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.643333 (+ 573us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.643461 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:25:38.644734 16657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.630966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36420)
0809 03:25:38.631484 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.631536 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.632481 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.632972 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.632977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.633461 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.633875 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.633888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.635072 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.635074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.636149 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.636159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.636711 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.636714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.636968 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.637714 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.637742 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.639633 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.641493 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.641502 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.641513 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.641868 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.642196 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.642199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.642201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.642348 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.642772 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.642780 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.643097 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.643288 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.643612 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":162,"threads_started":1}
I20260809 03:25:38.644769 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.631000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52932 (local address 127.9.46.252:45927)
0809 03:25:38.631560 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.631569 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.631602 (+ 33us) server_negotiation.cc:407] Connection header received
0809 03:25:38.632466 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.632484 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.632776 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.632947 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.633866 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.635366 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.636481 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.637249 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.637385 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.640359 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.640365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.640378 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.640409 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.642314 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.642787 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.642792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.642793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.642898 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.643241 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.643246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.643251 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.643515 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.644279 (+ 764us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.644389 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.644525 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":251,"threads_started":1}
I20260809 03:25:38.646313 16655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.252@KRBTEST.COM for oryx/127.9.46.253@KRBTEST.COM
I20260809 03:25:38.655403 16657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.646839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:45835 (local address 127.0.0.1:49834)
0809 03:25:38.647367 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.647382 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.647458 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.648058 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.648061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.648233 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.648423 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.648429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.649383 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.649390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.649926 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.649934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.650412 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.650415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.650554 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.650977 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.650988 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.652233 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.654113 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.654115 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.654116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.654344 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.654585 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.654587 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.654588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.654663 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.655013 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.655016 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.655099 (+ 83us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.655186 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.655287 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452}
I20260809 03:25:38.655459 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.647102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49834 (local address 127.9.46.253:45835)
0809 03:25:38.647835 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.647846 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.647859 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:25:38.647897 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.647899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.647937 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.648044 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.648585 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.649267 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.650073 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.650295 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.650403 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.652367 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.652370 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.652372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.652396 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.654007 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.654429 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.654431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.654432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.654503 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.654741 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.654743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.654744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.654912 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.654988 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.655244 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.655339 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":127,"threads_started":1}
I20260809 03:25:38.656500 16655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.252@KRBTEST.COM for oryx/127.9.46.252@KRBTEST.COM
I20260809 03:25:38.664975 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.657206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.9.46.252:45927)
0809 03:25:38.657313 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.657316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.657327 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:25:38.657354 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.657356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.657388 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.657452 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.658056 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.658989 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.659830 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.660105 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.660184 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.662077 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.662080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.662082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.662102 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.663463 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.663883 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.663885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.663886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.663955 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.664210 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.664212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.664213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.664365 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.664438 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.664746 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.664840 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":44}
I20260809 03:25:38.664995 16657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.657037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:45927 (local address 127.0.0.1:52946)
0809 03:25:38.657166 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.657180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.657256 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.657484 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.657486 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.657817 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.657928 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.657934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.659159 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.659162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.659717 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.659725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.660211 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.660215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.660278 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.660786 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.660797 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.661953 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.663592 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.663595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.663596 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.663788 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.664051 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.664053 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.664054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.664132 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.664445 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.664447 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.664532 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.664689 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.664789 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 03:25:38.668232 16655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: Bootstrap starting.
I20260809 03:25:38.668809 16098 heartbeater.cc:503] Master 127.9.46.254:43197 was elected leader, sending a full tablet report...
I20260809 03:25:38.671051 16655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:38.672071 16655 log.cc:824] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:38.674044 16655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: No bootstrap required, opened a new log
I20260809 03:25:38.677460 16655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } }
I20260809 03:25:38.677666 16655 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:38.677701 16655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a099fbf430442e8f90bbaf5dc69d92, State: Initialized, Role: FOLLOWER
I20260809 03:25:38.678118 16655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } }
I20260809 03:25:38.678622 16661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } } }
I20260809 03:25:38.678759 16661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:25:38.679291 16655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 03:25:38.685243 16672 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:25:38.685341 16672 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:25:38.689111 16497 master_runner.cc:428] Detected that this master 32a099fbf430442e8f90bbaf5dc69d92 is joining an existing cluster
I20260809 03:25:38.689188 16497 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.46.252:45927
I20260809 03:25:38.698832 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 15962
I20260809 03:25:38.712136 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:34455
--local_ip_for_outbound_sockets=127.9.46.193
--tserver_master_addrs=127.9.46.254:43197
--webserver_port=34793
--webserver_interface=127.9.46.193
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab
--principal=oryx/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.46.254:43197,127.9.46.253:45835
--tserver_master_addrs=127.9.46.254:43197,127.9.46.253:45835,127.9.46.252:45927 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:25:38.717818 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.691833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.9.46.254:43197)
0809 03:25:38.692014 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.692018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.692275 (+ 257us) server_negotiation.cc:407] Connection header received
0809 03:25:38.692483 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.692486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.692563 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.692624 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.693513 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.694374 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.696173 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.696877 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.696972 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.713857 (+ 16885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.713862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.713865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.713896 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.715745 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.716280 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.716284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.716286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.716402 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.716757 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.716759 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.716761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.717009 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.717082 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.717503 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.717651 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:25:38.718736 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.691369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36426)
0809 03:25:38.692184 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.692203 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.692363 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.692771 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.692774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.693197 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.693386 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.693393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.694505 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.694507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.696036 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.696046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.697123 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.697126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.697236 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.713111 (+ 15875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.713129 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.713643 (+ 514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.715912 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.715915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.715918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.716162 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.716523 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.716526 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.716528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.716650 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.717159 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.717162 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.717295 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.718362 (+ 1067us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.718534 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":3,"thread_start_us":387,"threads_started":1}
I20260809 03:25:38.720045 16679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.692114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:45835 (local address 127.0.0.1:49844)
0809 03:25:38.693197 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.693210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.693386 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.693743 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.693745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.694002 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.694197 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.694205 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.695342 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.695345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.696224 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.696232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.696965 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.696969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.697093 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.697598 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.697609 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.715676 (+ 18067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.717956 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.717962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.717966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.718244 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.718585 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.718591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.718596 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.718711 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.719284 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.719288 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.719416 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.719561 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.719755 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":30,"spinlock_wait_cycles":2688,"thread_start_us":99,"threads_started":1}
I20260809 03:25:38.720695 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.692476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49844 (local address 127.9.46.253:45835)
0809 03:25:38.692649 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.692653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.693311 (+ 658us) server_negotiation.cc:407] Connection header received
0809 03:25:38.693504 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.693507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.693559 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.693610 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.694332 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.695198 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.696339 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.696703 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.696785 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.715884 (+ 19099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.715890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.715895 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.715933 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.717816 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.718369 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.718375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.718380 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.718472 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.718835 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.718841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.718845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.719124 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.719230 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.720282 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.720445 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 03:25:38.723246 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.691649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52958 (local address 127.9.46.252:45927)
0809 03:25:38.691788 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.691793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.692616 (+ 823us) server_negotiation.cc:407] Connection header received
0809 03:25:38.692759 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.692762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.692801 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.692879 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.693640 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.694386 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.695544 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.695878 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.695957 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.697472 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.697478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.697486 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.697511 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.715130 (+ 17619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.721196 (+ 6066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.721202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.721206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.721298 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.721953 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.721958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.721961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.722223 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.722322 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.722890 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.723021 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:25:38.723857 16681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.691508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:45927 (local address 127.0.0.1:52958)
0809 03:25:38.692502 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.692517 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.692672 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.692987 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.692991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.693257 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.693460 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.693468 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.694512 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.694515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.695422 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.695433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.696113 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.696116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.696256 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.696810 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.696830 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.697315 (+ 485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.715255 (+ 17940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.715260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.715263 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.721053 (+ 5790us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.721714 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.721719 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.721724 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.721847 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.722379 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.722384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.722670 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.723444 (+ 774us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.723605 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"mutex_wait_us":46,"spinlock_wait_cycles":2944,"thread_start_us":413,"threads_started":1}
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245937, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.254@KRBTEST.COM for oryx/127.9.46.252@KRBTEST.COM
I20260809 03:25:38.733956 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.723930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:45927 (local address 127.0.0.1:52962)
0809 03:25:38.724156 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.724170 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.724243 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.725031 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.725034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.725227 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.725332 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.725340 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.726497 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.726504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.727508 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.727522 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.728119 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.728123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.728230 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.728770 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.728786 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.730328 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.732305 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.732309 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.732311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.732612 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.732909 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.732912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.732914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.732996 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.733437 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.733440 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.733535 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.733632 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.733781 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260809 03:25:38.734560 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.724375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52962 (local address 127.9.46.252:45927)
0809 03:25:38.724708 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.724712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.724726 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:25:38.724793 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.724795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.724845 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.724917 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.725470 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.726341 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.727643 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.727982 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.728094 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.730493 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.730497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.730500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.730525 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.732174 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.732725 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.732728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.732730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.732813 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.733101 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.733104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.733106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.733309 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.733393 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.734302 (+ 909us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.734417 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260809 03:25:38.735504 15897 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.46.252:45927: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92"
member_type: NON_VOTER
last_known_addr {
host: "127.9.46.252"
port: 45927
}
attrs {
promote: true
}
}
dest_uuid: "6d8ac0b2b6324b258134eff073a6aa4f"
cas_config_opid_index: 5
I20260809 03:25:38.736159 15897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } }
I20260809 03:25:38.737550 16484 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.739053 16484 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.46.252:45927
I20260809 03:25:38.739230 16484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.739342 16484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:38.740345 16497 master.cc:537] Master@127.9.46.252:45927 shutting down...
I20260809 03:25:38.740383 16628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:38.741894 16628 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 FOLLOWER]: Refusing update from remote peer 6d8ac0b2b6324b258134eff073a6aa4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 03:25:38.742795 16485 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:25:38.744875 15881 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f -> Peer 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252:45927): Couldn't send request to peer 32a099fbf430442e8f90bbaf5dc69d92. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 03:25:38.747287 16497 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:38.747574 16497 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:38.747681 16497 tablet_replica.cc:330] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: stopping tablet replica
I20260809 03:25:38.749867 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.739553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49848 (local address 127.9.46.253:45835)
0809 03:25:38.739710 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.739716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.741160 (+ 1444us) server_negotiation.cc:407] Connection header received
0809 03:25:38.741216 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.741219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.741277 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.741362 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:38.741730 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.742528 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.748128 (+ 5600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.748842 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.748950 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.749107 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.749643 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.749735 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:25:38.750420 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.737414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:45835 (local address 127.0.0.1:49848)
0809 03:25:38.740967 (+ 3553us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.740984 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.741070 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.741489 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.741493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.741510 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:38.741608 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.741614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.744029 (+ 2415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.744034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.747963 (+ 3929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.747976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.749175 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.749179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.749273 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.749326 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.750209 (+ 883us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.750297 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3475}
I20260809 03:25:38.751546 16541 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 LEARNER]: Refusing update from remote peer 6d8ac0b2b6324b258134eff073a6aa4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 03:25:38.752040 16485 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 03:25:38.752924 16497 master.cc:552] Master@127.9.46.252:45927 shutdown complete.
I20260809 03:25:38.752965 16497 master_runner.cc:305] Clearing existing system tablet
I20260809 03:25:38.753216 16497 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:38.753264 16497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:38.753969 16541 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.754472 16497 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 03:25:38.755256 16685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.755419 16497 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:38.755517 16497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:25:38.755545 16497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal
uuid: "32a099fbf430442e8f90bbaf5dc69d92"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "1e5a9e4faaf5de3cd1542e9281df4032"
server_key_iv: "d45d7ab33b662bc3082c43530afcc0fd"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.755808 16572 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d8ac0b2b6324b258134eff073a6aa4f. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.755919 16572 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: This master's current role is: LEARNER
I20260809 03:25:38.755988 16572 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: NON_VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: true } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.756048 16572 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: This master's current role is: LEARNER
I20260809 03:25:38.757537 16497 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:38.757604 16485 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f: attempting to promote NON_VOTER e00171ca5d8d4df38c465537e5906d59 to VOTER
I20260809 03:25:38.758041 16485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } }
I20260809 03:25:38.759928 16541 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 LEARNER]: Refusing update from remote peer 6d8ac0b2b6324b258134eff073a6aa4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 03:25:38.760181 16497 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 03:25:38.760272 16497 log.cc:1196] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 03:25:38.760351 16484 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 03:25:38.760509 16497 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: Deleting consensus metadata
I20260809 03:25:38.760711 16497 master_runner.cc:315] Copying system tablet from 127.9.46.254:43197
I20260809 03:25:38.761672 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.759405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:45927 (local address 127.0.0.1:52964)
0809 03:25:38.761502 (+ 2097us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.761594 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:45927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2015}
I20260809 03:25:38.761857 16497 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: tablet copy: Beginning tablet copy session from remote peer at address 127.9.46.254:43197
I20260809 03:25:38.763731 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.763227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:45927 (local address 127.0.0.1:52976)
0809 03:25:38.763554 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.763623 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:45927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":103,"threads_started":1}
W20260809 03:25:38.763830 15881 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f -> Peer 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252:45927): Couldn't send request to peer 32a099fbf430442e8f90bbaf5dc69d92. Status: Network error: Client connection negotiation failed: client connection to 127.9.46.252:45927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 03:25:38.764245 16485 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e00171ca5d8d4df38c465537e5906d59 (127.9.46.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.765779 16485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.765885 16485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:38.766001 16485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.766074 16485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:38.766763 16541 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e00171ca5d8d4df38c465537e5906d59 (127.9.46.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.768431 16686 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.768591 16686 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:25:38.777437 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.766207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36442 (local address 127.9.46.254:43197)
0809 03:25:38.766332 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.766336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.768799 (+ 2463us) server_negotiation.cc:407] Connection header received
0809 03:25:38.768912 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.768916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.768967 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.769052 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.769554 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.770387 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.771563 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.771884 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.771989 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.773317 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.773321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.773323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.773350 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.775338 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.775862 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.775866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.775868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.775963 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.776311 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.776316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.776318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.776529 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.776747 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.777154 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.777280 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:25:38.778049 16696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.765046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36442)
0809 03:25:38.768701 (+ 3655us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.768719 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.768830 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.769093 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.769096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.769320 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.769428 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.769435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.770521 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.770525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.771432 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.771443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.772007 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.772011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.772103 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.772721 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.772736 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.773168 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.775467 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.775470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.775473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.775753 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.776093 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.776096 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.776099 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.776202 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.776825 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.776828 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.776930 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.777701 (+ 771us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.777861 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1451,"thread_start_us":67,"threads_started":1}
I20260809 03:25:38.778658 15918 tablet_copy_service.cc:140] P 6d8ac0b2b6324b258134eff073a6aa4f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 32a099fbf430442e8f90bbaf5dc69d92 ({username='oryx', principal='oryx/127.9.46.252@KRBTEST.COM'} at 127.0.0.1:36442)
I20260809 03:25:38.778755 15918 tablet_copy_service.cc:161] P 6d8ac0b2b6324b258134eff073a6aa4f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 32a099fbf430442e8f90bbaf5dc69d92 at {username='oryx', principal='oryx/127.9.46.252@KRBTEST.COM'} at 127.0.0.1:36442: session id = 32a099fbf430442e8f90bbaf5dc69d92-00000000000000000000000000000000
I20260809 03:25:38.779874 15918 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 03:25:38.781351 16497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 03:25:38.784024 16497 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: tablet copy: Starting download of 0 data blocks...
I20260809 03:25:38.784222 16497 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: tablet copy: Starting download of 1 WAL segments...
I20260809 03:25:38.785928 16497 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 03:25:38.787878 15918 tablet_copy_service.cc:342] P 6d8ac0b2b6324b258134eff073a6aa4f: Request end of tablet copy session 32a099fbf430442e8f90bbaf5dc69d92-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.46.252@KRBTEST.COM'} at 127.0.0.1:36442
I20260809 03:25:38.787958 15918 tablet_copy_service.cc:433] P 6d8ac0b2b6324b258134eff073a6aa4f: ending tablet copy session 32a099fbf430442e8f90bbaf5dc69d92-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 32a099fbf430442e8f90bbaf5dc69d92
I20260809 03:25:38.795396 16497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:38.797106 16697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:38.797159 16701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:38.797597 16497 server_base.cc:1034] running on GCE node
W20260809 03:25:38.800683 16698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:38.800987 16497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:38.801323 16497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:38.802470 16497 hybrid_clock.cc:648] HybridClock initialized: now 1786245938802447 us; error 42 us; skew 500 ppm
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:38.804150 16497 init.cc:377] Logged in from keytab as oryx/127.9.46.252@KRBTEST.COM (short username oryx)
I20260809 03:25:38.804644 16497 webserver.cc:460] Webserver started at http://127.9.46.252:41491/ using document root <none> and password file <none>
I20260809 03:25:38.804836 16497 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:38.804876 16497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:38.806203 16497 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:25:38.807281 16708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.807586 16497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 03:25:38.807664 16497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal
uuid: "32a099fbf430442e8f90bbaf5dc69d92"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "1e5a9e4faaf5de3cd1542e9281df4032"
server_key_iv: "d45d7ab33b662bc3082c43530afcc0fd"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.807749 16497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:38.833846 16497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:38.836122 16497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:38.841583 16497 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.252:45927
I20260809 03:25:38.841624 16761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.252:45927 every 8 connection(s)
I20260809 03:25:38.842191 16497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
I20260809 03:25:38.844071 16762 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:25:38.845219 16762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: Bootstrap starting.
I20260809 03:25:38.857124 16762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:25:38.857463 16762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: Bootstrap complete.
I20260809 03:25:38.858121 16762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } }
I20260809 03:25:38.858266 16762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 32a099fbf430442e8f90bbaf5dc69d92, State: Initialized, Role: LEARNER
I20260809 03:25:38.858386 16762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } }
I20260809 03:25:38.858752 16768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:38.858870 16762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:38.858959 16768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: This master's current role is: LEARNER
W20260809 03:25:38.864348 16682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:38.864640 16682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:38.864743 16682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 03:25:38.868800 16780 catalog_manager.cc:1259] Loaded cluster ID: a64fda5661794ba1a078bff84af4bc4a
I20260809 03:25:38.868868 16780 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: loading cluster ID for follower catalog manager: success
W20260809 03:25:38.869714 16682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:38.869796 16682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:38.869910 16682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:38.871245 16780 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: acquiring CA information for follower catalog manager: success
I20260809 03:25:38.872797 16780 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:25:38.875945 16682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab
--principal=oryx/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:34455
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=34793
--tserver_master_addrs=127.9.46.254:43197,127.9.46.253:45835,127.9.46.252:45927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:38.877151 16682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:38.878186 16682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:25:38.885020 16238 heartbeater.cc:503] Master 127.9.46.254:43197 was elected leader, sending a full tablet report...
W20260809 03:25:38.888712 16783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:38.888695 16785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:38.888695 16782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:38.888732 16682 server_base.cc:1034] running on GCE node
I20260809 03:25:38.889564 16682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:38.890245 16682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:38.891436 16682 hybrid_clock.cc:648] HybridClock initialized: now 1786245938891401 us; error 49 us; skew 500 ppm
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:38.894654 16682 init.cc:377] Logged in from keytab as oryx/127.9.46.193@KRBTEST.COM (short username oryx)
I20260809 03:25:38.895493 16682 webserver.cc:460] Webserver started at http://127.9.46.193:34793/ using document root <none> and password file <none>
I20260809 03:25:38.896010 16682 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:38.896091 16682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:38.900043 16682 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:25:38.902334 16792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.903672 16682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:38.903782 16682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "ffd2a2de7b574340948f0136a92d236e"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "949640eae4143f95ab7b96ea97900a18"
server_key_iv: "c25e5f56d7a082fa0d3f57fc6fd729aa"
server_key_version: "encryptionkey@0"
I20260809 03:25:38.904345 16682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:38.927707 16682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:38.931260 16682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:38.931489 16682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:38.932194 16682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:38.933090 16682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:38.933163 16682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.933231 16682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:38.933264 16682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:38.944104 16682 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:34455
I20260809 03:25:38.944146 16910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:34455 every 8 connection(s)
I20260809 03:25:38.945215 16682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:38.952987 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 16682
I20260809 03:25:38.953240 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 16103
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.252@KRBTEST.COM
I20260809 03:25:38.963557 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.949213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:55671 (local address 127.9.46.254:43197)
0809 03:25:38.949346 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.949351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.949507 (+ 156us) server_negotiation.cc:407] Connection header received
0809 03:25:38.950071 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.950073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.950153 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.950231 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.951764 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.952598 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.954146 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.954494 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.954573 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.958780 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.958784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.958786 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.958817 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.960905 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.961599 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.961603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.961605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.961713 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.962176 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.962179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.962181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.962419 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.962503 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.963188 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.963323 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:25:38.964022 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.947574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:40707 (local address 127.9.46.252:45927)
0809 03:25:38.947885 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.947900 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.948638 (+ 738us) server_negotiation.cc:407] Connection header received
0809 03:25:38.949955 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.949962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.950021 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.950114 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.951764 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.952843 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.954257 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.954840 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.954956 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.958805 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.958813 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.958819 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.958858 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.961303 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.962073 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.962079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.962084 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.962202 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.962614 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.962620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.962625 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.962937 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.963048 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.963543 (+ 495us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.963726 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":126,"threads_started":1}
I20260809 03:25:38.965205 16919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.949009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.193:55671)
0809 03:25:38.949405 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.949421 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.949741 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.950346 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.950348 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.950905 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.951614 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.951622 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.952737 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.952743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.954016 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.954029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.954767 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.954772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.954975 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.955787 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.955811 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.958549 (+ 2738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.961070 (+ 2521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.961076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.961093 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.961465 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.961843 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.961846 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.961848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.962056 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.962597 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.962603 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.962913 (+ 310us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.963729 (+ 816us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.964098 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":223,"spinlock_wait_cycles":33408,"thread_start_us":197,"threads_started":1}
I20260809 03:25:38.965693 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.947994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:45927 (local address 127.9.46.193:40707)
0809 03:25:38.948456 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.948508 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.949741 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.950303 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.950314 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.950902 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.951612 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.951622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.952976 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.952979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.954088 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.954103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.955166 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.955173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.955386 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.956126 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.956149 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.958572 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.961559 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.961566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.961571 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.961910 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.962346 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.962352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.962357 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.962473 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.963146 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.963151 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.963290 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.964657 (+ 1367us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.965050 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":106,"spinlock_wait_cycles":20992,"thread_start_us":130,"threads_started":1}
I20260809 03:25:38.966657 16912 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43197
I20260809 03:25:38.966820 16911 heartbeater.cc:347] Connected to a master server at 127.9.46.252:45927
I20260809 03:25:38.966960 16912 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:38.966958 16911 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:38.967687 16912 heartbeater.cc:511] Master 127.9.46.254:43197 requested a full tablet report, sending...
Aug 09 03:25:38 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245938, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.253@KRBTEST.COM
I20260809 03:25:38.968477 16911 heartbeater.cc:511] Master 127.9.46.252:45927 requested a full tablet report, sending...
I20260809 03:25:38.969796 15897 ts_manager.cc:194] Re-registered known tserver with Master: ffd2a2de7b574340948f0136a92d236e (127.9.46.193:34455)
I20260809 03:25:38.969923 16718 ts_manager.cc:194] Registered new tserver with Master: ffd2a2de7b574340948f0136a92d236e (127.9.46.193:34455)
I20260809 03:25:38.970528 15897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:55671
I20260809 03:25:38.973147 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.947142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:39233 (local address 127.9.46.253:45835)
0809 03:25:38.947336 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:38.947342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:38.948632 (+ 1290us) server_negotiation.cc:407] Connection header received
0809 03:25:38.950043 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:38.950049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:38.950115 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:38.950167 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:38.951747 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.952280 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.964657 (+ 12377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.964992 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.965084 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.968494 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:38.968498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:38.968510 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:38.968563 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:38.971432 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.971875 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.971879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.971880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.971972 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:38.972266 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:38.972269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:38.972271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:38.972456 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:38.972643 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:38.972881 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 03:25:38.972997 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 03:25:38.973172 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:38.947994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:45835 (local address 127.9.46.193:39233)
0809 03:25:38.948482 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:38.948533 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:38.949739 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:38.950323 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:38.950325 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:38.950899 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:38.951600 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.951615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.952428 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.952432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.964461 (+ 12029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:38.964472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:38.965271 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:38.965275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:38.965436 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:38.966381 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:38.966396 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:38.968302 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:38.971555 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.971561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.971563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.971765 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.972071 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:38.972075 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:38.972077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:38.972166 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:38.972596 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:38.972602 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:38.972695 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 03:25:38.972814 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 03:25:38.972980 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":150,"spinlock_wait_cycles":31104,"thread_start_us":201,"threads_started":1}
I20260809 03:25:38.974822 16914 heartbeater.cc:347] Connected to a master server at 127.9.46.253:45835
I20260809 03:25:38.974905 16914 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:38.975127 16914 heartbeater.cc:511] Master 127.9.46.253:45835 requested a full tablet report, sending...
I20260809 03:25:38.976133 16520 ts_manager.cc:194] Registered new tserver with Master: ffd2a2de7b574340948f0136a92d236e (127.9.46.193:34455)
I20260809 03:25:38.980901 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:43137
--local_ip_for_outbound_sockets=127.9.46.194
--tserver_master_addrs=127.9.46.254:43197
--webserver_port=44273
--webserver_interface=127.9.46.194
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab
--principal=oryx/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.46.254:43197,127.9.46.253:45835
--tserver_master_addrs=127.9.46.254:43197,127.9.46.253:45835,127.9.46.252:45927 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:39.096271 16920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:39.096560 16920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:39.096652 16920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 03:25:39.097414 16379 heartbeater.cc:503] Master 127.9.46.254:43197 was elected leader, sending a full tablet report...
W20260809 03:25:39.101166 16920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:39.101248 16920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:39.101332 16920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:39.106535 16920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab
--principal=oryx/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:43137
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=44273
--tserver_master_addrs=127.9.46.254:43197,127.9.46.253:45835,127.9.46.252:45927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:39.107780 16920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:39.108953 16920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:39.118652 16926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:39.118654 16928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:39.118697 16925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:39.119001 16920 server_base.cc:1034] running on GCE node
I20260809 03:25:39.119426 16920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:39.120061 16920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:39.121279 16920 hybrid_clock.cc:648] HybridClock initialized: now 1786245939121260 us; error 30 us; skew 500 ppm
Aug 09 03:25:39 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245939, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:39.124208 16920 init.cc:377] Logged in from keytab as oryx/127.9.46.194@KRBTEST.COM (short username oryx)
I20260809 03:25:39.125351 16920 webserver.cc:460] Webserver started at http://127.9.46.194:44273/ using document root <none> and password file <none>
I20260809 03:25:39.125973 16920 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:39.126108 16920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:39.129891 16920 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 03:25:39.131938 16935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:39.133268 16920 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:39.133409 16920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "41c925bbe2c34b3db10dca58dca3ec1b"
format_stamp: "Formatted at 2026-08-09 03:25:37 on dist-test-slave-65mx"
server_key: "862abe6af4b60112b2c3a13642f971bb"
server_key_iv: "5597b5dddf54f6b0c2032545cda632db"
server_key_version: "encryptionkey@0"
I20260809 03:25:39.133948 16920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:39.152894 16920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:39.155987 16920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:39.156203 16920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:39.156870 16920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:39.157755 16920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:39.157827 16920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:39.157899 16920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:39.157953 16920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:39.167583 16920 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:43137
I20260809 03:25:39.167616 17055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:43137 every 8 connection(s)
I20260809 03:25:39.168692 16920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:39.178326 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 16920
I20260809 03:25:39.178563 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 16244
Aug 09 03:25:39 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245939, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.253@KRBTEST.COM
Aug 09 03:25:39 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245939, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.252@KRBTEST.COM
I20260809 03:25:39.189284 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.172462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:58761 (local address 127.9.46.253:45835)
0809 03:25:39.172659 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.172663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.172886 (+ 223us) server_negotiation.cc:407] Connection header received
0809 03:25:39.173332 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.173336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.173394 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.173477 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.174819 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.175347 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.177067 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.177389 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.177461 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.180940 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.180947 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.180952 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.180984 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.182935 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.187259 (+ 4324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.187265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.187270 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.187401 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.187836 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.187842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.187847 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.188189 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.188298 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.188970 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.189114 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 03:25:39.190371 17063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.172345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:45835 (local address 127.9.46.194:58761)
0809 03:25:39.172785 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.172799 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.173132 (+ 333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.173673 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.173680 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.174119 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:39.174669 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.174683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.175496 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.175500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.176917 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.176927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.177640 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.177643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.177758 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.178500 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:39.178517 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:39.180756 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:39.183076 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.183084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.183101 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.187094 (+ 3993us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.187497 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.187500 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.187502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.187705 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.188299 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:39.188307 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:39.188713 (+ 406us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.188910 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.189557 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":128,"thread_start_us":254,"threads_started":1}
I20260809 03:25:39.192340 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.170556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:52887 (local address 127.9.46.252:45927)
0809 03:25:39.170694 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.170700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.171985 (+ 1285us) server_negotiation.cc:407] Connection header received
0809 03:25:39.173365 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.173369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.173441 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.173497 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.174841 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.175649 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.176867 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.177179 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.177299 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.188491 (+ 11192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.188497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.188504 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.188575 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.190575 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.191131 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.191134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.191135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.191215 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.191523 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.191525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.191526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.191697 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.191770 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.192117 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.192231 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:25:39.192533 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.171148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:45927 (local address 127.9.46.194:52887)
0809 03:25:39.171799 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.171864 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.173120 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.173679 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.173682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.174148 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:39.174671 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.174685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.175795 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.175798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.176665 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.176675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.177336 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.177340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.177544 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.178608 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:39.178623 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:39.188327 (+ 9704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:39.190716 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.190722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.190727 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.190994 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.191313 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.191319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.191324 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.191427 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.191794 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:39.191800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:39.191921 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.192037 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.192218 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":176,"spinlock_wait_cycles":60544,"thread_start_us":196,"threads_started":1}
I20260809 03:25:39.193915 17056 heartbeater.cc:347] Connected to a master server at 127.9.46.252:45927
Aug 09 03:25:39 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245939, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:39.194113 17058 heartbeater.cc:347] Connected to a master server at 127.9.46.253:45835
I20260809 03:25:39.194171 17056 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:39.194204 17058 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:39.194674 17058 heartbeater.cc:511] Master 127.9.46.253:45835 requested a full tablet report, sending...
I20260809 03:25:39.194677 17056 heartbeater.cc:511] Master 127.9.46.252:45927 requested a full tablet report, sending...
I20260809 03:25:39.195894 16520 ts_manager.cc:194] Registered new tserver with Master: 41c925bbe2c34b3db10dca58dca3ec1b (127.9.46.194:43137)
I20260809 03:25:39.195935 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:35363
--local_ip_for_outbound_sockets=127.9.46.195
--tserver_master_addrs=127.9.46.254:43197
--webserver_port=44009
--webserver_interface=127.9.46.195
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab
--principal=oryx/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.46.254:43197,127.9.46.253:45835
--tserver_master_addrs=127.9.46.254:43197,127.9.46.253:45835,127.9.46.252:45927 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:25:39.196878 16718 ts_manager.cc:194] Registered new tserver with Master: 41c925bbe2c34b3db10dca58dca3ec1b (127.9.46.194:43137)
I20260809 03:25:39.198391 17062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.171148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.194:37959)
0809 03:25:39.171799 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.171864 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.173120 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.173690 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.173692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.174147 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:39.174671 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.174685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.175784 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.175789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.176843 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.176857 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.177452 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.177455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.177563 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.178315 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:39.178339 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:39.194452 (+ 16113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:39.196691 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.196697 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.196702 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.196987 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.197320 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.197326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.197331 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.197436 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.197828 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:39.197831 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:39.197934 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.198037 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.198196 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":220,"spinlock_wait_cycles":25088,"thread_start_us":219,"threads_started":1}
I20260809 03:25:39.198410 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.170566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:37959 (local address 127.9.46.254:43197)
0809 03:25:39.170694 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.170700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.171985 (+ 1285us) server_negotiation.cc:407] Connection header received
0809 03:25:39.173447 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.173450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.173507 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.173576 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.174839 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.175648 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.177011 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.177350 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.177459 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.194640 (+ 17181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.194646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.194651 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.194683 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.196522 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.197123 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.197130 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.197135 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.197234 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.197543 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.197545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.197546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.197725 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.197834 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.198152 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.198273 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:25:39.198971 17057 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43197
I20260809 03:25:39.199076 17057 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:39.199268 17057 heartbeater.cc:511] Master 127.9.46.254:43197 requested a full tablet report, sending...
I20260809 03:25:39.199865 15897 ts_manager.cc:194] Re-registered known tserver with Master: 41c925bbe2c34b3db10dca58dca3ec1b (127.9.46.194:43137)
I20260809 03:25:39.200577 15897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:37959
I20260809 03:25:39.301564 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.297319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52984 (local address 127.9.46.252:45927)
0809 03:25:39.297505 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.297510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.297783 (+ 273us) server_negotiation.cc:407] Connection header received
0809 03:25:39.297916 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.297920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.297998 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.298120 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:39.298365 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.299208 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.300228 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.301012 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.301126 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.301294 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.301349 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.301419 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 03:25:39.301517 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.297211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:45927 (local address 127.0.0.1:52984)
0809 03:25:39.297679 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.297692 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.297806 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.298135 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.298137 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.298152 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:39.298249 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.298253 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.299322 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.299325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.300101 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.300110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.301162 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.301165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.301242 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.301267 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.301319 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.301384 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":130,"threads_started":1}
I20260809 03:25:39.302440 16730 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 03:25:39.302851 16768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d8ac0b2b6324b258134eff073a6aa4f. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:39.303072 16768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: This master's current role is: LEARNER
W20260809 03:25:39.315191 17064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:39.315428 17064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:39.315508 17064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:39.319543 17064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:39.319600 17064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:39.319687 17064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:39.325199 17064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab
--principal=oryx/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:35363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=44009
--tserver_master_addrs=127.9.46.254:43197,127.9.46.253:45835,127.9.46.252:45927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:39.326339 17064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:39.327469 17064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:39.338088 17073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:39.338177 17064 server_base.cc:1034] running on GCE node
W20260809 03:25:39.338101 17070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:39.338104 17071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:39.338824 17064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:39.339504 17064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:39.340641 17064 hybrid_clock.cc:648] HybridClock initialized: now 1786245939340609 us; error 44 us; skew 500 ppm
Aug 09 03:25:39 dist-test-slave-65mx krb5kdc[15851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245939, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:39.343537 17064 init.cc:377] Logged in from keytab as oryx/127.9.46.195@KRBTEST.COM (short username oryx)
I20260809 03:25:39.344403 17064 webserver.cc:460] Webserver started at http://127.9.46.195:44009/ using document root <none> and password file <none>
I20260809 03:25:39.345008 17064 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:39.345094 17064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:39.349094 17064 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:25:39.351318 17080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:39.353065 17064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:39.353211 17064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "dfde0bfae5c34f62914fb7a50e962a74"
format_stamp: "Formatted at 2026-08-09 03:25:38 on dist-test-slave-65mx"
server_key: "d0ed23d8a712fca73e17d928e5f769b6"
server_key_iv: "7c5966600dc3c6376e0061b31e93bebb"
server_key_version: "encryptionkey@0"
I20260809 03:25:39.353809 17064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:39.382845 17064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:39.386097 17064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:39.386310 17064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:39.387042 17064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:39.387987 17064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:39.388059 17064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:39.388132 17064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:39.388183 17064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:39.399180 17064 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:35363
I20260809 03:25:39.399207 17198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:35363 every 8 connection(s)
I20260809 03:25:39.400275 17064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:39.404786 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 17064
Aug 09 03:25:39 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245939, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
Aug 09 03:25:39 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245939, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.252@KRBTEST.COM
Aug 09 03:25:39 dist-test-slave-65mx krb5kdc[15851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245939, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.253@KRBTEST.COM
I20260809 03:25:39.420696 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.402222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:42175 (local address 127.9.46.252:45927)
0809 03:25:39.402353 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.402358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.403289 (+ 931us) server_negotiation.cc:407] Connection header received
0809 03:25:39.404436 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.404440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.404505 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.404583 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.406648 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.407489 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.409749 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.410119 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.410202 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.415669 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.415674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.415679 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.415715 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.417861 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.418538 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.418541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.418543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.418638 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.419123 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.419128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.419132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.419402 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.419485 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.420305 (+ 820us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.420447 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:25:39.422470 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.402207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:36209 (local address 127.9.46.253:45835)
0809 03:25:39.402363 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.402369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.403286 (+ 917us) server_negotiation.cc:407] Connection header received
0809 03:25:39.404408 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.404412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.404471 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.404582 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.411861 (+ 7279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.412731 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.413950 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.414302 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.414415 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.417170 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.417174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.417177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.417206 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.419993 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.420677 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.420681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.420683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.420774 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.421110 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.421114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.421116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.421320 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.421405 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.422159 (+ 754us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.422281 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64,"spinlock_wait_cycles":2048}
I20260809 03:25:39.422602 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.403795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:42017 (local address 127.9.46.254:43197)
0809 03:25:39.403914 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.403919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.404002 (+ 83us) server_negotiation.cc:407] Connection header received
0809 03:25:39.404437 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.404440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.404499 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.404590 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.406648 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.407471 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.409749 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.410119 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.410202 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.418399 (+ 8197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.418404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.418407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.418442 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.420690 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.421217 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.421221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.421223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.421318 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.421650 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.421653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.421655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.421865 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.421936 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.422288 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.422412 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:25:39.423116 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.403656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.195:42017)
0809 03:25:39.403926 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.403939 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.404238 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.404886 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.404897 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.405561 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:39.406471 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.406490 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.407702 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.407705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.409594 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.409605 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.410419 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.410423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.410825 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.411623 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:39.411644 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:39.415476 (+ 3832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:39.420825 (+ 5349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.420829 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.420832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.421103 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.421428 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.421431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.421432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.421537 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.422009 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:39.422013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:39.422102 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.422750 (+ 648us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.422899 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":585,"spinlock_wait_cycles":2944,"thread_start_us":107,"threads_started":1}
I20260809 03:25:39.423291 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.402656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:45835 (local address 127.9.46.195:36209)
0809 03:25:39.403129 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.403183 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.404218 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.404889 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.404899 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.405618 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:39.411703 (+ 6085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.411716 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.412879 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.412884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.413817 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.413827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.414787 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.414792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.414900 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.415472 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:39.415486 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:39.417004 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:39.420126 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.420129 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.420131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.420412 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.420886 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.420888 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.420890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.421002 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.421448 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:39.421450 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:39.421619 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.421717 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.421855 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":79,"spinlock_wait_cycles":78080,"thread_start_us":145,"threads_started":1}
I20260809 03:25:39.424196 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.402656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:45927 (local address 127.9.46.195:42175)
0809 03:25:39.403128 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.403182 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.404234 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.404955 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.404958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.405599 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:39.406471 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.406490 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.407702 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.407705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.409594 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.409605 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.410419 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.410423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.410791 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.413306 (+ 2515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:39.413321 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:39.415025 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:39.418015 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.418022 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.418040 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.418405 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.418776 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:39.418780 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:39.418783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:39.418987 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:39.419585 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:39.419592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:39.419957 (+ 365us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.421279 (+ 1322us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.421746 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":478,"spinlock_wait_cycles":25344,"thread_start_us":185,"threads_started":1}
I20260809 03:25:39.424954 17200 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43197
I20260809 03:25:39.424989 17201 heartbeater.cc:347] Connected to a master server at 127.9.46.253:45835
I20260809 03:25:39.425230 17201 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:39.425232 17200 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:39.425880 17201 heartbeater.cc:511] Master 127.9.46.253:45835 requested a full tablet report, sending...
I20260809 03:25:39.426597 17200 heartbeater.cc:511] Master 127.9.46.254:43197 requested a full tablet report, sending...
I20260809 03:25:39.426783 17199 heartbeater.cc:347] Connected to a master server at 127.9.46.252:45927
I20260809 03:25:39.426856 17199 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:39.427042 17199 heartbeater.cc:511] Master 127.9.46.252:45927 requested a full tablet report, sending...
I20260809 03:25:39.427415 16520 ts_manager.cc:194] Registered new tserver with Master: dfde0bfae5c34f62914fb7a50e962a74 (127.9.46.195:35363)
I20260809 03:25:39.427713 16718 ts_manager.cc:194] Registered new tserver with Master: dfde0bfae5c34f62914fb7a50e962a74 (127.9.46.195:35363)
I20260809 03:25:39.428580 15897 ts_manager.cc:194] Re-registered known tserver with Master: dfde0bfae5c34f62914fb7a50e962a74 (127.9.46.195:35363)
I20260809 03:25:39.429342 15897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:42017
I20260809 03:25:39.430188 17212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.409117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.9.46.253:45835)
0809 03:25:39.410172 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.410203 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.411955 (+ 1752us) server_negotiation.cc:407] Connection header received
0809 03:25:39.411994 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.411996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.412044 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.412152 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.414967 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.417758 (+ 2791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.421629 (+ 3871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.421978 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.422083 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.425921 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.425925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.425928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.425956 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.427866 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.428389 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.428393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.428395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.428497 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.428896 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.428899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.428901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.429104 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.429274 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.429883 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.430014 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"spinlock_wait_cycles":3840,"thread_start_us":106,"threads_started":1}
I20260809 03:25:39.431273 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.407844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36444 (local address 127.9.46.254:43197)
0809 03:25:39.410806 (+ 2962us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.410812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.410828 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:25:39.410868 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.410874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.410928 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.411017 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.413196 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.413985 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.420256 (+ 6271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.422297 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.422398 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.424426 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.424429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.424432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.424469 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.429054 (+ 4585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.429605 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.429608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.429611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.429713 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.430036 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.430039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.430041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.430257 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.430381 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.430987 (+ 606us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.431120 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2873,"thread_start_us":112,"threads_started":1}
I20260809 03:25:39.436028 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.416573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.9.46.252:45927)
0809 03:25:39.420888 (+ 4315us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.420892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.426038 (+ 5146us) server_negotiation.cc:407] Connection header received
0809 03:25:39.426301 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.426304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.426356 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.426413 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.427434 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.428244 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.430625 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.430934 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.431032 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.432699 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.432704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.432708 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.432739 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.434338 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.434774 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.434777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.434779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.434868 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.435166 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.435169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.435171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.435356 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.435496 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.435740 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.435861 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4243,"thread_start_us":117,"threads_started":1}
I20260809 03:25:39.434405 15897 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:39.437795 15897 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:39.440507 16688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: NON_VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: true } } }
I20260809 03:25:39.440712 16688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:39.443742 17219 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f: attempting to promote NON_VOTER 32a099fbf430442e8f90bbaf5dc69d92 to VOTER
I20260809 03:25:39.444100 17219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } }
I20260809 03:25:39.445365 16730 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 LEARNER]: Refusing update from remote peer 6d8ac0b2b6324b258134eff073a6aa4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 03:25:39.445425 16541 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Refusing update from remote peer 6d8ac0b2b6324b258134eff073a6aa4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 03:25:39.445792 16688 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 03:25:39.445919 17219 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 03:25:39.447702 17219 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:39.448132 16541 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:39.448469 16730 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:39.449662 17218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:39.449795 17218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:39.449996 17218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:39.450094 17218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:39.450271 17218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:39.450356 17218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:39.450335 16768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:39.450436 16768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:25:39.451849 17227 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:39.451996 17227 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:25:39.462513 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.457271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:35363 (local address 127.0.0.1:57376)
0809 03:25:39.457380 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.457397 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.457499 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.458962 (+ 1463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.458964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.458982 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:39.459124 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.459134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.460118 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.460120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.461128 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.461137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.462124 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.462128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.462209 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.462231 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.462299 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.462374 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 03:25:39.462541 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.457531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57376 (local address 127.9.46.195:35363)
0809 03:25:39.458326 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.458340 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.458366 (+ 26us) server_negotiation.cc:407] Connection header received
0809 03:25:39.458452 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.458468 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.458709 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.459077 (+ 368us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:39.459250 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.460016 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.461257 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.461987 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.462118 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.462275 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.462329 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.462417 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":229,"threads_started":1}
I20260809 03:25:39.465245 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.458050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:43137 (local address 127.0.0.1:39268)
0809 03:25:39.460289 (+ 2239us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.460301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.460410 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.460922 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.460925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.460945 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:39.461033 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.461038 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.462400 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.462403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.463507 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.463518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.464827 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.464830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.464917 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.464941 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.465016 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.465100 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2164,"thread_start_us":114,"threads_started":1}
I20260809 03:25:39.465865 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.458489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.9.46.194:43137)
0809 03:25:39.459018 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.459031 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.460393 (+ 1362us) server_negotiation.cc:407] Connection header received
0809 03:25:39.460531 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.460561 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.460770 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.460927 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:39.461149 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.462251 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.463672 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.464472 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.464613 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.464759 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.465620 (+ 861us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.465726 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":141,"threads_started":1}
I20260809 03:25:39.466096 17132 tablet_service.cc:1467] Processing CreateTablet for tablet 13dd6bf276a7443294ed003184b91c15 (DEFAULT_TABLE table=test-table [id=30ddbcad9b4a43c7a065519e423ae6a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:39.466616 17231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.457794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:34455 (local address 127.0.0.1:51176)
0809 03:25:39.458634 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.458649 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.458737 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.461478 (+ 2741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.461482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.461502 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:39.461589 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.461595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.463970 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.463974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.465011 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.465021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.466187 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.466195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.466294 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.466323 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.466414 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.466490 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":118,"threads_started":1}
I20260809 03:25:39.466698 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.457972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.9.46.193:34455)
0809 03:25:39.460696 (+ 2724us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.460713 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.460734 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:25:39.460805 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.460810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.461304 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.461793 (+ 489us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:39.461930 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.463856 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.465122 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.465821 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.466003 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.466223 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.466473 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.466563 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2618,"thread_start_us":119,"threads_started":1}
I20260809 03:25:39.467257 17132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13dd6bf276a7443294ed003184b91c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:39.469332 16844 tablet_service.cc:1467] Processing CreateTablet for tablet 13dd6bf276a7443294ed003184b91c15 (DEFAULT_TABLE table=test-table [id=30ddbcad9b4a43c7a065519e423ae6a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:39.470232 16844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13dd6bf276a7443294ed003184b91c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:39.475629 16988 tablet_service.cc:1467] Processing CreateTablet for tablet 13dd6bf276a7443294ed003184b91c15 (DEFAULT_TABLE table=test-table [id=30ddbcad9b4a43c7a065519e423ae6a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:39.476130 17234 tablet_bootstrap.cc:492] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: Bootstrap starting.
I20260809 03:25:39.476763 16988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13dd6bf276a7443294ed003184b91c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:39.477047 17235 tablet_bootstrap.cc:492] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: Bootstrap starting.
I20260809 03:25:39.479532 17235 tablet_bootstrap.cc:654] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:39.479808 17234 tablet_bootstrap.cc:654] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:39.480867 17235 log.cc:824] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:39.480979 17234 log.cc:824] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:39.483078 17235 tablet_bootstrap.cc:492] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: No bootstrap required, opened a new log
I20260809 03:25:39.483078 17234 tablet_bootstrap.cc:492] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: No bootstrap required, opened a new log
I20260809 03:25:39.483286 17235 ts_tablet_manager.cc:1397] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:25:39.483286 17234 ts_tablet_manager.cc:1397] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 03:25:39.488339 17235 raft_consensus.cc:348] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } }
I20260809 03:25:39.488680 17235 raft_consensus.cc:374] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:39.488744 17235 raft_consensus.cc:729] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd2a2de7b574340948f0136a92d236e, State: Initialized, Role: FOLLOWER
I20260809 03:25:39.489027 17234 raft_consensus.cc:348] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } }
I20260809 03:25:39.489286 17234 raft_consensus.cc:374] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:39.489342 17234 raft_consensus.cc:729] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfde0bfae5c34f62914fb7a50e962a74, State: Initialized, Role: FOLLOWER
I20260809 03:25:39.489320 17235 consensus_queue.cc:260] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } }
I20260809 03:25:39.489753 17234 consensus_queue.cc:260] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } }
I20260809 03:25:39.490473 17235 ts_tablet_manager.cc:1428] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 03:25:39.490619 17234 ts_tablet_manager.cc:1428] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260809 03:25:39.490983 17200 heartbeater.cc:503] Master 127.9.46.254:43197 was elected leader, sending a full tablet report...
I20260809 03:25:39.491010 16912 heartbeater.cc:503] Master 127.9.46.254:43197 was elected leader, sending a full tablet report...
I20260809 03:25:39.491180 17238 tablet_bootstrap.cc:492] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: Bootstrap starting.
I20260809 03:25:39.493844 17238 tablet_bootstrap.cc:654] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:39.494781 17238 log.cc:824] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:39.497550 17238 tablet_bootstrap.cc:492] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: No bootstrap required, opened a new log
I20260809 03:25:39.497728 17238 ts_tablet_manager.cc:1397] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 03:25:39.501681 17238 raft_consensus.cc:348] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } }
I20260809 03:25:39.501901 17238 raft_consensus.cc:374] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:39.501957 17238 raft_consensus.cc:729] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c925bbe2c34b3db10dca58dca3ec1b, State: Initialized, Role: FOLLOWER
I20260809 03:25:39.502373 17238 consensus_queue.cc:260] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } }
I20260809 03:25:39.503074 17057 heartbeater.cc:503] Master 127.9.46.254:43197 was elected leader, sending a full tablet report...
I20260809 03:25:39.503433 17238 ts_tablet_manager.cc:1428] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:25:39.651337 17240 raft_consensus.cc:482] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:39.651572 17240 raft_consensus.cc:504] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } }
W20260809 03:25:39.652275 17202 tablet.cc:2369] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:39.652993 17240 leader_election.cc:288] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41c925bbe2c34b3db10dca58dca3ec1b (127.9.46.194:43137), ffd2a2de7b574340948f0136a92d236e (127.9.46.193:34455)
I20260809 03:25:39.656714 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.653180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:43137 (local address 127.9.46.195:43287)
0809 03:25:39.653358 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.653374 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.653470 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.653739 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.653742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.653759 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:39.653995 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.654003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.654810 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.654812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.655572 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.655580 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.656276 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.656280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.656365 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.656381 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.656476 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.656566 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":2}
I20260809 03:25:39.656701 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.653476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:38885 (local address 127.9.46.193:34455)
0809 03:25:39.653617 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.653620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.653632 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:25:39.653700 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.653706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.653758 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.653837 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:39.654164 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.654649 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.655701 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.656084 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.656175 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.656249 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.656463 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.656569 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:25:39.656724 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.653315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:43287 (local address 127.9.46.194:43137)
0809 03:25:39.653443 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.653448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.653489 (+ 41us) server_negotiation.cc:407] Connection header received
0809 03:25:39.653555 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.653558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.653616 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.653701 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:39.654129 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.654672 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.655731 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.656148 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.656265 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.656315 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.656519 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.656616 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:25:39.656772 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.653281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:34455 (local address 127.9.46.195:38885)
0809 03:25:39.653471 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:39.653484 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:39.653566 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:39.653862 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:39.653864 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:39.653879 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:39.654015 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.654020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.654759 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.654766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.655562 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:39.655571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.656204 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.656209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:39.656312 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.656345 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:25:39.656415 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 03:25:39.656516 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":2}
I20260809 03:25:39.657446 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13dd6bf276a7443294ed003184b91c15" candidate_uuid: "dfde0bfae5c34f62914fb7a50e962a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd2a2de7b574340948f0136a92d236e" is_pre_election: true
I20260809 03:25:39.657491 17009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13dd6bf276a7443294ed003184b91c15" candidate_uuid: "dfde0bfae5c34f62914fb7a50e962a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" is_pre_election: true
I20260809 03:25:39.657907 16864 raft_consensus.cc:2454] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfde0bfae5c34f62914fb7a50e962a74 in term 0.
I20260809 03:25:39.657898 17009 raft_consensus.cc:2454] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfde0bfae5c34f62914fb7a50e962a74 in term 0.
I20260809 03:25:39.658452 17084 leader_election.cc:302] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 41c925bbe2c34b3db10dca58dca3ec1b, dfde0bfae5c34f62914fb7a50e962a74, ffd2a2de7b574340948f0136a92d236e; no voters:
I20260809 03:25:39.658807 17240 raft_consensus.cc:2781] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:39.658905 17240 raft_consensus.cc:482] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:39.658962 17240 raft_consensus.cc:3037] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:39.660172 17240 raft_consensus.cc:504] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } }
I20260809 03:25:39.660523 17240 leader_election.cc:288] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [CANDIDATE]: Term 1 election: Requested vote from peers 41c925bbe2c34b3db10dca58dca3ec1b (127.9.46.194:43137), ffd2a2de7b574340948f0136a92d236e (127.9.46.193:34455)
I20260809 03:25:39.660974 17009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13dd6bf276a7443294ed003184b91c15" candidate_uuid: "dfde0bfae5c34f62914fb7a50e962a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c925bbe2c34b3db10dca58dca3ec1b"
I20260809 03:25:39.660991 16864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13dd6bf276a7443294ed003184b91c15" candidate_uuid: "dfde0bfae5c34f62914fb7a50e962a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffd2a2de7b574340948f0136a92d236e"
I20260809 03:25:39.661089 17009 raft_consensus.cc:3037] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:39.661119 16864 raft_consensus.cc:3037] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:39.662133 17009 raft_consensus.cc:2454] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfde0bfae5c34f62914fb7a50e962a74 in term 1.
I20260809 03:25:39.662248 16864 raft_consensus.cc:2454] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfde0bfae5c34f62914fb7a50e962a74 in term 1.
I20260809 03:25:39.662415 17083 leader_election.cc:302] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c925bbe2c34b3db10dca58dca3ec1b, dfde0bfae5c34f62914fb7a50e962a74; no voters:
I20260809 03:25:39.662617 17240 raft_consensus.cc:2781] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:39.662848 17240 raft_consensus.cc:686] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 1 LEADER]: Becoming Leader. State: Replica: dfde0bfae5c34f62914fb7a50e962a74, State: Running, Role: LEADER
I20260809 03:25:39.663189 17240 consensus_queue.cc:237] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } }
I20260809 03:25:39.666705 15896 catalog_manager.cc:5495] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfde0bfae5c34f62914fb7a50e962a74 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "dfde0bfae5c34f62914fb7a50e962a74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } health_report { overall_health: UNKNOWN } } }
W20260809 03:25:39.671147 17060 tablet.cc:2369] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:25:39.696923 16915 tablet.cc:2369] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:39.699481 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.695916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.9.46.195:35363)
0809 03:25:39.696059 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.696073 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.696088 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:25:39.696193 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.696195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.696251 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.696361 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:39.696648 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.697230 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.698090 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.698303 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.698410 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.698727 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:39.699117 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:39.699249 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.699325 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.699375 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:25:39.705382 17009 raft_consensus.cc:1261] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b [term 1 FOLLOWER]: Refusing update from remote peer dfde0bfae5c34f62914fb7a50e962a74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:39.705426 16864 raft_consensus.cc:1261] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e [term 1 FOLLOWER]: Refusing update from remote peer dfde0bfae5c34f62914fb7a50e962a74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:39.706161 17240 consensus_queue.cc:1035] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:39.706429 17243 consensus_queue.cc:1035] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:39.725749 15896 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:39.725943 15896 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:39.730693 15896 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d8ac0b2b6324b258134eff073a6aa4f: Sending DeleteTablet for 3 replicas of tablet 13dd6bf276a7443294ed003184b91c15
I20260809 03:25:39.731614 16988 tablet_service.cc:1514] Processing DeleteTablet for tablet 13dd6bf276a7443294ed003184b91c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:39 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:39268
I20260809 03:25:39.731658 17132 tablet_service.cc:1514] Processing DeleteTablet for tablet 13dd6bf276a7443294ed003184b91c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:39 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:57376
I20260809 03:25:39.732148 17256 tablet_replica.cc:330] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: stopping tablet replica
I20260809 03:25:39.732197 17257 tablet_replica.cc:330] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: stopping tablet replica
I20260809 03:25:39.732594 17257 raft_consensus.cc:2229] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:39.732780 17256 raft_consensus.cc:2229] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:39.732968 17257 raft_consensus.cc:2258] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:39.733412 17256 raft_consensus.cc:2258] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:39.734480 16844 tablet_service.cc:1514] Processing DeleteTablet for tablet 13dd6bf276a7443294ed003184b91c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:39 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:51176
I20260809 03:25:39.735034 17257 ts_tablet_manager.cc:1905] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:39.735188 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 15865
I20260809 03:25:39.738641 17256 ts_tablet_manager.cc:1905] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:39.739990 17257 ts_tablet_manager.cc:1918] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:39.740123 17257 log.cc:1196] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/13dd6bf276a7443294ed003184b91c15
I20260809 03:25:39.740525 17257 ts_tablet_manager.cc:1939] T 13dd6bf276a7443294ed003184b91c15 P dfde0bfae5c34f62914fb7a50e962a74: Deleting consensus metadata
I20260809 03:25:39.742589 17256 ts_tablet_manager.cc:1918] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:39.742714 17256 log.cc:1196] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/13dd6bf276a7443294ed003184b91c15
I20260809 03:25:39.743055 17256 ts_tablet_manager.cc:1939] T 13dd6bf276a7443294ed003184b91c15 P 41c925bbe2c34b3db10dca58dca3ec1b: Deleting consensus metadata
W20260809 03:25:39.745958 16939 connection.cc:535] server connection from 127.0.0.1:39268 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 03:25:39.746294 17084 connection.cc:535] server connection from 127.0.0.1:57376 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:25:39.748821 17259 tablet_replica.cc:330] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: stopping tablet replica
I20260809 03:25:39.749269 17259 raft_consensus.cc:2229] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:39.749688 17259 raft_consensus.cc:2258] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:39.751760 17259 ts_tablet_manager.cc:1905] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:39.755244 17259 ts_tablet_manager.cc:1918] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:39.755370 17259 log.cc:1196] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/13dd6bf276a7443294ed003184b91c15
I20260809 03:25:39.755752 17259 ts_tablet_manager.cc:1939] T 13dd6bf276a7443294ed003184b91c15 P ffd2a2de7b574340948f0136a92d236e: Deleting consensus metadata
W20260809 03:25:39.757084 16796 connection.cc:441] server connection from 127.0.0.1:51176 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51176 (request call id 1) could send its response
I20260809 03:25:39.760684 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.750739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49876 (local address 127.9.46.253:45835)
0809 03:25:39.750870 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.750874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.751288 (+ 414us) server_negotiation.cc:407] Connection header received
0809 03:25:39.751396 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.751399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.751453 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.751548 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.752169 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.752984 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.754209 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.754541 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.754613 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.756239 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.756243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.756246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.756272 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.758042 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.758577 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.758582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.758584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.758672 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.759600 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.759603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.759605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.759835 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.759937 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.760375 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.760489 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:25:39.762632 17212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.733854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49872 (local address 127.9.46.253:45835)
0809 03:25:39.733992 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.733997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.744789 (+ 10792us) server_negotiation.cc:407] Connection header received
0809 03:25:39.744988 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.744991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.745055 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.745150 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.746205 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.746983 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.748191 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.748619 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.748736 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.751812 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.751816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.751819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.751848 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.753872 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.757258 (+ 3386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.757263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.757267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.757367 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.757690 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.757694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.757696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.757938 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.758079 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.759270 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.759794 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:25:39.765060 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.750088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.9.46.252:45927)
0809 03:25:39.751367 (+ 1279us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.751372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.751387 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:25:39.751432 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.751435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.751500 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.751590 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.755535 (+ 3945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.756314 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.757874 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.758234 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.758317 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.760125 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.760131 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.760137 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.760173 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.762864 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.763562 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.763566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.763567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.763668 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.763998 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.764002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.764005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.764259 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.764414 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.764703 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.764849 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1176}
I20260809 03:25:39.768432 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:39.733744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52996 (local address 127.9.46.252:45927)
0809 03:25:39.733990 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:39.733995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:39.759008 (+ 25013us) server_negotiation.cc:407] Connection header received
0809 03:25:39.759199 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:39.759203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:39.759258 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:39.759336 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:39.760666 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.761487 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.762624 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:39.762971 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:39.763091 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:39.764820 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:39.764825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:39.764828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:39.764850 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:39.766659 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.767059 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.767063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.767064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.767148 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:39.767469 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:39.767473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:39.767474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:39.767702 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:39.767811 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:39.768153 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 03:25:39.768282 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260809 03:25:40.722422 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:40.721762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.193:46217)
0809 03:25:40.722184 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:40.722298 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":90,"threads_started":1}
W20260809 03:25:40.722749 16912 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:43197 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:40.724377 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:40.723993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.195:42029)
0809 03:25:40.724224 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:40.724299 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":71,"threads_started":1}
W20260809 03:25:40.724758 17200 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:43197 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:40.725260 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:40.724880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.194:52407)
0809 03:25:40.725108 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:40.725181 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":64,"threads_started":1}
W20260809 03:25:40.725551 17057 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:43197 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:41.231020 17271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6d8ac0b2b6324b258134eff073a6aa4f)
I20260809 03:25:41.231019 17270 raft_consensus.cc:482] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6d8ac0b2b6324b258134eff073a6aa4f)
I20260809 03:25:41.231253 17271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } }
I20260809 03:25:41.231252 17270 raft_consensus.cc:504] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } }
I20260809 03:25:41.232192 17271 leader_election.cc:288] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197), e00171ca5d8d4df38c465537e5906d59 (127.9.46.253:45835)
I20260809 03:25:41.232214 17270 leader_election.cc:288] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197), 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252:45927)
I20260809 03:25:41.232786 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.232290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36882)
0809 03:25:41.232547 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.232677 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":59,"threads_started":1}
W20260809 03:25:41.233050 16513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:41.233153 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.232287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36874)
0809 03:25:41.232980 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.233059 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":65,"threads_started":1}
W20260809 03:25:41.233322 16711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:41.234244 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.233969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36892)
0809 03:25:41.234109 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.234163 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 03:25:41.234395 16711 leader_election.cc:334] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197): Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:41.234549 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.233916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36888)
0809 03:25:41.234370 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.234428 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260809 03:25:41.234761 16513 leader_election.cc:334] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197): Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:41.236871 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.232486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:45927 (local address 127.0.0.1:53006)
0809 03:25:41.232775 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.232791 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:41.232894 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:41.233365 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:41.233370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:41.233391 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:41.233544 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:41.233551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:41.234839 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:41.234843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:41.235667 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:41.235677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:41.236354 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:41.236359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:41.236484 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:41.236510 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 03:25:41.236652 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 03:25:41.236731 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260809 03:25:41.236936 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.232732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53006 (local address 127.9.46.252:45927)
0809 03:25:41.233096 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:41.233101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:41.233115 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:25:41.233160 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:41.233166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:41.233225 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:41.233649 (+ 424us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:41.233750 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:41.234673 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:41.235800 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:41.236218 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:41.236375 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:41.236494 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:41.236667 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 03:25:41.236810 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":294,"thread_start_us":89,"threads_started":1}
I20260809 03:25:41.236896 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.232913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49884 (local address 127.9.46.253:45835)
0809 03:25:41.233158 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:41.233162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:41.233472 (+ 310us) server_negotiation.cc:407] Connection header received
0809 03:25:41.233513 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:41.233516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:41.233552 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:41.233640 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:41.234022 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:41.234765 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:41.235773 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:41.236204 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:41.236327 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:41.236454 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:41.236692 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 03:25:41.236750 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":183,"thread_start_us":63,"threads_started":1}
I20260809 03:25:41.237121 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.232628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:45835 (local address 127.0.0.1:49884)
0809 03:25:41.233310 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.233324 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:41.233398 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:41.233732 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:41.233734 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:41.233749 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:41.233899 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:41.233905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:41.234859 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:41.234862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:41.235647 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:41.235655 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:41.236341 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:41.236345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:41.236523 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:41.236588 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 03:25:41.236666 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:25:41.236840 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":154,"thread_start_us":76,"threads_started":1}
I20260809 03:25:41.237350 16730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e00171ca5d8d4df38c465537e5906d59" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32a099fbf430442e8f90bbaf5dc69d92" is_pre_election: true
I20260809 03:25:41.237550 16541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32a099fbf430442e8f90bbaf5dc69d92" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e00171ca5d8d4df38c465537e5906d59" is_pre_election: true
I20260809 03:25:41.237607 16730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e00171ca5d8d4df38c465537e5906d59 in term 1.
I20260809 03:25:41.237759 16541 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32a099fbf430442e8f90bbaf5dc69d92 in term 1.
I20260809 03:25:41.238090 16511 leader_election.cc:302] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32a099fbf430442e8f90bbaf5dc69d92, e00171ca5d8d4df38c465537e5906d59; no voters: 6d8ac0b2b6324b258134eff073a6aa4f
I20260809 03:25:41.238134 16711 leader_election.cc:302] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32a099fbf430442e8f90bbaf5dc69d92, e00171ca5d8d4df38c465537e5906d59; no voters: 6d8ac0b2b6324b258134eff073a6aa4f
I20260809 03:25:41.238335 17270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 03:25:41.238366 17271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 03:25:41.238394 17270 raft_consensus.cc:482] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6d8ac0b2b6324b258134eff073a6aa4f)
I20260809 03:25:41.238420 17271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6d8ac0b2b6324b258134eff073a6aa4f)
I20260809 03:25:41.238449 17270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:25:41.238466 17271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:25:41.239444 17270 raft_consensus.cc:504] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } }
I20260809 03:25:41.239446 17271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } }
I20260809 03:25:41.239756 17271 leader_election.cc:288] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [CANDIDATE]: Term 2 election: Requested vote from peers 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197), e00171ca5d8d4df38c465537e5906d59 (127.9.46.253:45835)
I20260809 03:25:41.239799 17270 leader_election.cc:288] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 2 election: Requested vote from peers 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197), 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252:45927)
I20260809 03:25:41.240108 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.239884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36896)
0809 03:25:41.239984 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.240038 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 03:25:41.240150 16730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e00171ca5d8d4df38c465537e5906d59" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32a099fbf430442e8f90bbaf5dc69d92"
I20260809 03:25:41.240321 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.240109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36904)
0809 03:25:41.240199 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.240252 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 03:25:41.240345 16730 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e00171ca5d8d4df38c465537e5906d59 in current term 2: Already voted for candidate 32a099fbf430442e8f90bbaf5dc69d92 in this term.
I20260809 03:25:41.240454 16541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32a099fbf430442e8f90bbaf5dc69d92" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e00171ca5d8d4df38c465537e5906d59"
I20260809 03:25:41.240651 16541 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 32a099fbf430442e8f90bbaf5dc69d92 in current term 2: Already voted for candidate e00171ca5d8d4df38c465537e5906d59 in this term.
I20260809 03:25:41.240828 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.240595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36918)
0809 03:25:41.240721 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.240772 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 03:25:41.240907 16513 leader_election.cc:334] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197): Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:41.240967 16513 leader_election.cc:302] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e00171ca5d8d4df38c465537e5906d59; no voters: 32a099fbf430442e8f90bbaf5dc69d92, 6d8ac0b2b6324b258134eff073a6aa4f
I20260809 03:25:41.241117 17270 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 03:25:41.241236 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.241002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:36932)
0809 03:25:41.241127 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.241179 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 03:25:41.241462 16711 leader_election.cc:334] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197): Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:41.241590 16711 leader_election.cc:302] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32a099fbf430442e8f90bbaf5dc69d92; no voters: 6d8ac0b2b6324b258134eff073a6aa4f, e00171ca5d8d4df38c465537e5906d59
I20260809 03:25:41.241743 17271 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 03:25:41.723716 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.723280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.193:34983)
0809 03:25:41.723565 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.723627 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":101,"threads_started":1}
I20260809 03:25:41.725648 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.725272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.195:42583)
0809 03:25:41.725518 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.725572 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":86,"threads_started":1}
I20260809 03:25:41.726445 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:41.726098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.194:54507)
0809 03:25:41.726315 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:41.726371 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":72,"threads_started":1}
I20260809 03:25:42.725152 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:42.724726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.193:57175)
0809 03:25:42.724992 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:42.725054 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
I20260809 03:25:42.726658 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:42.726281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.195:60095)
0809 03:25:42.726517 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:42.726577 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
I20260809 03:25:42.727486 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:42.727135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.194:45301)
0809 03:25:42.727356 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:42.727409 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":67,"threads_started":1}
I20260809 03:25:42.949545 17290 raft_consensus.cc:482] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:42.949694 17290 raft_consensus.cc:504] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } }
I20260809 03:25:42.950207 17290 leader_election.cc:288] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197), 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252:45927)
I20260809 03:25:42.950729 16730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e00171ca5d8d4df38c465537e5906d59" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32a099fbf430442e8f90bbaf5dc69d92" is_pre_election: true
I20260809 03:25:42.950866 16730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e00171ca5d8d4df38c465537e5906d59 in term 2.
I20260809 03:25:42.950933 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:42.950431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:37138)
0809 03:25:42.950744 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:42.950800 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":90,"threads_started":1}
I20260809 03:25:42.951280 16511 leader_election.cc:302] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a099fbf430442e8f90bbaf5dc69d92, e00171ca5d8d4df38c465537e5906d59; no voters:
I20260809 03:25:42.951485 17290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 03:25:42.951543 17290 raft_consensus.cc:482] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:42.951571 17290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 2 FOLLOWER]: Advancing to term 3
I20260809 03:25:42.951889 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:42.951577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:37150)
0809 03:25:42.951762 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:42.951820 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260809 03:25:42.952075 16513 leader_election.cc:334] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197): Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:42.952877 17290 raft_consensus.cc:504] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } }
I20260809 03:25:42.953238 17290 leader_election.cc:288] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 3 election: Requested vote from peers 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197), 32a099fbf430442e8f90bbaf5dc69d92 (127.9.46.252:45927)
I20260809 03:25:42.953642 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:42.953372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:37152)
0809 03:25:42.953494 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:42.953548 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 03:25:42.953642 16730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e00171ca5d8d4df38c465537e5906d59" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32a099fbf430442e8f90bbaf5dc69d92"
I20260809 03:25:42.953814 16730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 2 FOLLOWER]: Advancing to term 3
I20260809 03:25:42.954284 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:42.954085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:37154)
0809 03:25:42.954184 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:42.954230 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 03:25:42.954385 16513 leader_election.cc:334] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197): Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
I20260809 03:25:42.954983 16730 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e00171ca5d8d4df38c465537e5906d59 in term 3.
I20260809 03:25:42.955307 16511 leader_election.cc:302] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32a099fbf430442e8f90bbaf5dc69d92, e00171ca5d8d4df38c465537e5906d59; no voters: 6d8ac0b2b6324b258134eff073a6aa4f
I20260809 03:25:42.955498 17290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 03:25:42.955673 17290 raft_consensus.cc:686] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [term 3 LEADER]: Becoming Leader. State: Replica: e00171ca5d8d4df38c465537e5906d59, State: Running, Role: LEADER
I20260809 03:25:42.955940 17290 consensus_queue.cc:237] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } }
I20260809 03:25:42.957160 17293 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e00171ca5d8d4df38c465537e5906d59. Latest consensus state: current_term: 3 leader_uuid: "e00171ca5d8d4df38c465537e5906d59" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:42.957300 17293 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:42.957695 17295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:42.960448 17295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:42.960950 17295 catalog_manager.cc:1259] Loaded cluster ID: a64fda5661794ba1a078bff84af4bc4a
I20260809 03:25:42.961036 17295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:42.961649 17295 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:42.962066 17295 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Loaded TSK: 0
I20260809 03:25:42.962788 17295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:25:42.989224 16529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:42.992334 16529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:42.995638 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:42.995311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:37158)
0809 03:25:42.995475 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:42.995549 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 03:25:42.995703 16730 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [term 3 FOLLOWER]: Refusing update from remote peer e00171ca5d8d4df38c465537e5906d59: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 03:25:42.996212 17293 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 03:25:42.996637 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:42.996326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.0.0.1:37172)
0809 03:25:42.996476 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:42.996524 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 03:25:42.996758 16513 consensus_peers.cc:487] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 -> Peer 6d8ac0b2b6324b258134eff073a6aa4f (127.9.46.254:43197): Couldn't send request to peer 6d8ac0b2b6324b258134eff073a6aa4f. Status: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 03:25:42.997851 17297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e00171ca5d8d4df38c465537e5906d59. Latest consensus state: current_term: 3 leader_uuid: "e00171ca5d8d4df38c465537e5906d59" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:42.997994 17297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:25:42.999166 17290 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e00171ca5d8d4df38c465537e5906d59" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:42.999339 17290 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:42.999660 17297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e00171ca5d8d4df38c465537e5906d59" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "6d8ac0b2b6324b258134eff073a6aa4f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43197 } } peers { permanent_uuid: "e00171ca5d8d4df38c465537e5906d59" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 45835 } attrs { promote: false } } peers { permanent_uuid: "32a099fbf430442e8f90bbaf5dc69d92" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 45927 } attrs { promote: false } } }
I20260809 03:25:42.999788 17297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a099fbf430442e8f90bbaf5dc69d92 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:25:43.010898 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.005771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.9.46.195:35363)
0809 03:25:43.006045 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:43.006050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:43.006146 (+ 96us) server_negotiation.cc:407] Connection header received
0809 03:25:43.006661 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:43.006664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:43.006720 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:43.006813 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:43.007102 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.007860 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.009361 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.010004 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.010116 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.010182 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:43.010690 (+ 508us) server_negotiation.cc:299] Negotiation successful
0809 03:25:43.010768 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":86,"threads_started":1}
I20260809 03:25:43.010897 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.005646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:35363 (local address 127.0.0.1:57388)
0809 03:25:43.006050 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.006065 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:43.006178 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:43.006834 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:43.006837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:43.006855 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:43.006962 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:43.006969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.007956 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.007960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:43.009016 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:43.009027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.010466 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.010473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:43.010572 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.010598 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 03:25:43.010673 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:25:43.010756 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":20}
I20260809 03:25:43.011185 17305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.006312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.9.46.193:34455)
0809 03:25:43.006583 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:43.006587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:43.006659 (+ 72us) server_negotiation.cc:407] Connection header received
0809 03:25:43.007133 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:43.007136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:43.007191 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:43.007276 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:43.007544 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.008345 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.009840 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.010432 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.010539 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.010608 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:43.010961 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 03:25:43.011054 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260809 03:25:43.011351 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.006459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39282 (local address 127.9.46.194:43137)
0809 03:25:43.006723 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:43.006741 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:43.007207 (+ 466us) server_negotiation.cc:407] Connection header received
0809 03:25:43.007273 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:43.007275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:43.007320 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:43.007404 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:43.007781 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.008665 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.010100 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.010551 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.010654 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.010723 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:43.011114 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 03:25:43.011191 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":67,"threads_started":1}
I20260809 03:25:43.011512 17132 tablet_service.cc:1467] Processing CreateTablet for tablet 5b2bea26c82e4464a643ca2f80a9e84d (DEFAULT_TABLE table=test-table [id=9ceca140cb4e4853b3921088be34750d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:43.011574 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.005846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:34455 (local address 127.0.0.1:51180)
0809 03:25:43.006571 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.006585 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:43.006732 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:43.007310 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:43.007313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:43.007332 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:43.007420 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:43.007425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.008485 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.008488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:43.009698 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:43.009709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.010568 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.010575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:43.010681 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.010710 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 03:25:43.010782 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 03:25:43.010874 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":6,"thread_start_us":101,"threads_started":1}
I20260809 03:25:43.011759 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.006112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:43137 (local address 127.0.0.1:39282)
0809 03:25:43.007023 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.007039 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:43.007126 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:43.007537 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:43.007539 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:43.007557 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:43.007648 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:43.007653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.008789 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.008792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:43.009974 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:43.009981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.010783 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.010787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:43.010898 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.010924 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 03:25:43.011479 (+ 555us) client_negotiation.cc:240] Negotiation successful
0809 03:25:43.011553 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":30,"thread_start_us":99,"threads_started":1}
I20260809 03:25:43.011881 17132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b2bea26c82e4464a643ca2f80a9e84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:43.012393 16988 tablet_service.cc:1467] Processing CreateTablet for tablet 5b2bea26c82e4464a643ca2f80a9e84d (DEFAULT_TABLE table=test-table [id=9ceca140cb4e4853b3921088be34750d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:43.012708 16988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b2bea26c82e4464a643ca2f80a9e84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:43.013234 16844 tablet_service.cc:1467] Processing CreateTablet for tablet 5b2bea26c82e4464a643ca2f80a9e84d (DEFAULT_TABLE table=test-table [id=9ceca140cb4e4853b3921088be34750d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:43.013496 16844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b2bea26c82e4464a643ca2f80a9e84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:43.014444 17307 tablet_bootstrap.cc:492] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74: Bootstrap starting.
I20260809 03:25:43.015513 17307 tablet_bootstrap.cc:654] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:43.015625 17308 tablet_bootstrap.cc:492] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b: Bootstrap starting.
I20260809 03:25:43.016139 17309 tablet_bootstrap.cc:492] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e: Bootstrap starting.
I20260809 03:25:43.016646 17308 tablet_bootstrap.cc:654] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:43.017095 17309 tablet_bootstrap.cc:654] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:43.017418 17307 tablet_bootstrap.cc:492] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74: No bootstrap required, opened a new log
I20260809 03:25:43.017522 17307 ts_tablet_manager.cc:1397] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:25:43.017874 17308 tablet_bootstrap.cc:492] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b: No bootstrap required, opened a new log
I20260809 03:25:43.017930 17307 raft_consensus.cc:348] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } }
I20260809 03:25:43.018042 17308 ts_tablet_manager.cc:1397] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.002s
I20260809 03:25:43.018075 17307 raft_consensus.cc:374] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:43.018149 17307 raft_consensus.cc:729] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfde0bfae5c34f62914fb7a50e962a74, State: Initialized, Role: FOLLOWER
I20260809 03:25:43.018306 17307 consensus_queue.cc:260] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } }
I20260809 03:25:43.018503 17309 tablet_bootstrap.cc:492] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e: No bootstrap required, opened a new log
I20260809 03:25:43.018527 17308 raft_consensus.cc:348] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } }
I20260809 03:25:43.018703 17307 ts_tablet_manager.cc:1428] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 03:25:43.019127 17309 ts_tablet_manager.cc:1397] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:25:43.019285 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.019037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.195:50491)
0809 03:25:43.019142 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.019206 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 03:25:43.019826 17309 raft_consensus.cc:348] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } }
I20260809 03:25:43.019944 17309 raft_consensus.cc:374] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:43.020018 17309 raft_consensus.cc:729] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd2a2de7b574340948f0136a92d236e, State: Initialized, Role: FOLLOWER
I20260809 03:25:43.020169 17309 consensus_queue.cc:260] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } }
I20260809 03:25:43.020610 17309 ts_tablet_manager.cc:1428] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:43.021418 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.021038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.193:48343)
0809 03:25:43.021288 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.021346 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260809 03:25:43.021883 17201 heartbeater.cc:503] Master 127.9.46.253:45835 was elected leader, sending a full tablet report...
I20260809 03:25:43.022200 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.021939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.193:57079)
0809 03:25:43.022077 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.022126 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 03:25:43.022568 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.022339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.195:59387)
0809 03:25:43.022445 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.022501 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 03:25:43.022786 17308 raft_consensus.cc:374] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:43.022908 17308 raft_consensus.cc:729] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c925bbe2c34b3db10dca58dca3ec1b, State: Initialized, Role: FOLLOWER
I20260809 03:25:43.023149 17308 consensus_queue.cc:260] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } }
I20260809 03:25:43.023929 17308 ts_tablet_manager.cc:1428] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20260809 03:25:43.024302 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.024060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.194:36631)
0809 03:25:43.024170 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.024228 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 03:25:43.024629 16914 heartbeater.cc:503] Master 127.9.46.253:45835 was elected leader, sending a full tablet report...
I20260809 03:25:43.025055 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.024810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.194:52293)
0809 03:25:43.024929 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.024986 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 03:25:43.026273 17058 heartbeater.cc:503] Master 127.9.46.253:45835 was elected leader, sending a full tablet report...
I20260809 03:25:43.147838 17316 raft_consensus.cc:482] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:43.148039 17316 raft_consensus.cc:504] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } }
I20260809 03:25:43.149331 17316 leader_election.cc:288] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfde0bfae5c34f62914fb7a50e962a74 (127.9.46.195:35363), 41c925bbe2c34b3db10dca58dca3ec1b (127.9.46.194:43137)
I20260809 03:25:43.153076 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.149623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:35363 (local address 127.9.46.193:33111)
0809 03:25:43.149779 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.149795 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:43.149920 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:43.150219 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:43.150222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:43.150241 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:43.150385 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:43.150390 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.151130 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.151134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:43.151944 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:43.151953 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.152638 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.152643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:43.152744 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.152773 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 03:25:43.152865 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 03:25:43.152936 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"spinlock_wait_cycles":1792}
I20260809 03:25:43.153076 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.149749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:33111 (local address 127.9.46.195:35363)
0809 03:25:43.149893 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:43.149898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:43.149913 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:25:43.150053 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:43.150057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:43.150111 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:43.150194 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:43.150499 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.151009 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.152071 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.152471 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.152638 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.152695 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:43.152880 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 03:25:43.152953 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 03:25:43.153462 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.149744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:52017 (local address 127.9.46.194:43137)
0809 03:25:43.149891 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:43.149898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:43.150255 (+ 357us) server_negotiation.cc:407] Connection header received
0809 03:25:43.150423 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:43.150426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:43.150484 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:43.150569 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:43.150847 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.151351 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.152373 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.152862 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.152992 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.153041 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:43.153270 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 03:25:43.153350 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:25:43.153643 17152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2bea26c82e4464a643ca2f80a9e84d" candidate_uuid: "ffd2a2de7b574340948f0136a92d236e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfde0bfae5c34f62914fb7a50e962a74" is_pre_election: true
I20260809 03:25:43.153863 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.149623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:43137 (local address 127.9.46.193:52017)
0809 03:25:43.150157 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.150169 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:43.150308 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:43.150580 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:43.150583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:43.150608 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:43.150736 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:43.150742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.151462 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.151467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:43.152247 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:43.152256 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.152984 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.152988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:43.153084 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.153106 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:25:43.153625 (+ 519us) client_negotiation.cc:240] Negotiation successful
0809 03:25:43.153701 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":184,"spinlock_wait_cycles":1920,"thread_start_us":148,"threads_started":1}
I20260809 03:25:43.153906 17152 raft_consensus.cc:2454] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffd2a2de7b574340948f0136a92d236e in term 0.
I20260809 03:25:43.154366 17009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2bea26c82e4464a643ca2f80a9e84d" candidate_uuid: "ffd2a2de7b574340948f0136a92d236e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" is_pre_election: true
I20260809 03:25:43.154382 16793 leader_election.cc:302] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfde0bfae5c34f62914fb7a50e962a74, ffd2a2de7b574340948f0136a92d236e; no voters:
I20260809 03:25:43.154518 17009 raft_consensus.cc:2454] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffd2a2de7b574340948f0136a92d236e in term 0.
I20260809 03:25:43.154677 17316 raft_consensus.cc:2781] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:43.154747 17316 raft_consensus.cc:482] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:43.154799 17316 raft_consensus.cc:3037] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:43.155850 17316 raft_consensus.cc:504] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } }
I20260809 03:25:43.156181 17316 leader_election.cc:288] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [CANDIDATE]: Term 1 election: Requested vote from peers dfde0bfae5c34f62914fb7a50e962a74 (127.9.46.195:35363), 41c925bbe2c34b3db10dca58dca3ec1b (127.9.46.194:43137)
I20260809 03:25:43.156512 17152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2bea26c82e4464a643ca2f80a9e84d" candidate_uuid: "ffd2a2de7b574340948f0136a92d236e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfde0bfae5c34f62914fb7a50e962a74"
I20260809 03:25:43.156582 17009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2bea26c82e4464a643ca2f80a9e84d" candidate_uuid: "ffd2a2de7b574340948f0136a92d236e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41c925bbe2c34b3db10dca58dca3ec1b"
I20260809 03:25:43.156673 17152 raft_consensus.cc:3037] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:43.156729 17009 raft_consensus.cc:3037] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:43.157718 17009 raft_consensus.cc:2454] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffd2a2de7b574340948f0136a92d236e in term 1.
I20260809 03:25:43.157792 17152 raft_consensus.cc:2454] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffd2a2de7b574340948f0136a92d236e in term 1.
I20260809 03:25:43.158035 16795 leader_election.cc:302] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41c925bbe2c34b3db10dca58dca3ec1b, ffd2a2de7b574340948f0136a92d236e; no voters:
I20260809 03:25:43.158233 17316 raft_consensus.cc:2781] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:43.158516 17316 raft_consensus.cc:686] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 1 LEADER]: Becoming Leader. State: Replica: ffd2a2de7b574340948f0136a92d236e, State: Running, Role: LEADER
I20260809 03:25:43.158789 17316 consensus_queue.cc:237] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } }
I20260809 03:25:43.159418 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.159173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.193:46559)
0809 03:25:43.159293 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.159349 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 03:25:43.161684 16529 catalog_manager.cc:5495] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e reported cstate change: term changed from 0 to 1, leader changed from <none> to ffd2a2de7b574340948f0136a92d236e (127.9.46.193). New cstate: current_term: 1 leader_uuid: "ffd2a2de7b574340948f0136a92d236e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffd2a2de7b574340948f0136a92d236e" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 34455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 } health_report { overall_health: UNKNOWN } } }
I20260809 03:25:43.205595 17305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.202253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.9.46.193:34455)
0809 03:25:43.202387 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:43.202391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:43.202406 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:25:43.202486 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:43.202487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:43.202545 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:43.202641 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:43.202873 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.203398 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.204194 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:43.204403 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:43.204482 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:43.204807 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:43.205262 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:43.205352 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:43.205427 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 03:25:43.205477 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:25:43.208771 17009 raft_consensus.cc:1261] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b [term 1 FOLLOWER]: Refusing update from remote peer ffd2a2de7b574340948f0136a92d236e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:43.208927 17152 raft_consensus.cc:1261] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74 [term 1 FOLLOWER]: Refusing update from remote peer ffd2a2de7b574340948f0136a92d236e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:43.209431 17316 consensus_queue.cc:1035] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c925bbe2c34b3db10dca58dca3ec1b" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:43.209684 17319 consensus_queue.cc:1035] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfde0bfae5c34f62914fb7a50e962a74" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:43.211572 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.211247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.194:54489)
0809 03:25:43.211414 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.211477 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 03:25:43.212747 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.212165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.195:34981)
0809 03:25:43.212619 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.212675 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364}
I20260809 03:25:43.214908 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.214561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.193:42047)
0809 03:25:43.214777 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.214833 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260809 03:25:43.216738 17327 mvcc.cc:205] Tried to move back new op lower bound from 7316463383376764928 to 7316463383181094912. Current Snapshot: MvccSnapshot[applied={T|T < 7316463383376764928}]
I20260809 03:25:43.216756 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.216466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.195:57731)
0809 03:25:43.216627 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.216681 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 03:25:43.220095 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.219805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.193:49571)
0809 03:25:43.219963 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.220019 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 03:25:43.220180 17325 mvcc.cc:205] Tried to move back new op lower bound from 7316463383376764928 to 7316463383181094912. Current Snapshot: MvccSnapshot[applied={T|T < 7316463383376764928}]
I20260809 03:25:43.221486 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.221117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43197 (local address 127.9.46.194:44761)
0809 03:25:43.221363 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.221416 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260809 03:25:43.223481 16520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:43.223687 16520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:43.227387 16520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e00171ca5d8d4df38c465537e5906d59: Sending DeleteTablet for 3 replicas of tablet 5b2bea26c82e4464a643ca2f80a9e84d
I20260809 03:25:43.228114 17132 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b2bea26c82e4464a643ca2f80a9e84d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:43 UTC) from {username='oryx', principal='oryx/127.9.46.253@KRBTEST.COM'} at 127.0.0.1:57388
I20260809 03:25:43.228176 16988 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b2bea26c82e4464a643ca2f80a9e84d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:43 UTC) from {username='oryx', principal='oryx/127.9.46.253@KRBTEST.COM'} at 127.0.0.1:39282
I20260809 03:25:43.228742 16844 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b2bea26c82e4464a643ca2f80a9e84d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:43 UTC) from {username='oryx', principal='oryx/127.9.46.253@KRBTEST.COM'} at 127.0.0.1:51180
I20260809 03:25:43.228783 17335 tablet_replica.cc:330] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b: stopping tablet replica
I20260809 03:25:43.228935 17335 raft_consensus.cc:2229] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:43.229079 17335 raft_consensus.cc:2258] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:43.229233 17336 tablet_replica.cc:330] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e: stopping tablet replica
I20260809 03:25:43.229399 17336 raft_consensus.cc:2229] T 5b2bea26c82e4464a643ca2f80a9e84d P ffd2a2de7b574340948f0136a92d236e [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:43.229622 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 16682
I20260809 03:25:43.230422 17335 ts_tablet_manager.cc:1905] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:43.232873 17335 ts_tablet_manager.cc:1918] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:43.232941 17335 log.cc:1196] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/5b2bea26c82e4464a643ca2f80a9e84d
I20260809 03:25:43.233253 17335 ts_tablet_manager.cc:1939] T 5b2bea26c82e4464a643ca2f80a9e84d P 41c925bbe2c34b3db10dca58dca3ec1b: Deleting consensus metadata
I20260809 03:25:43.233538 17334 tablet_replica.cc:330] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74: stopping tablet replica
I20260809 03:25:43.233669 17334 raft_consensus.cc:2229] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:43.233810 17334 raft_consensus.cc:2258] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:43.234037 16513 catalog_manager.cc:4841] TS 41c925bbe2c34b3db10dca58dca3ec1b (127.9.46.194:43137): tablet 5b2bea26c82e4464a643ca2f80a9e84d (table test-table [id=9ceca140cb4e4853b3921088be34750d]) successfully deleted
I20260809 03:25:43.234504 17334 ts_tablet_manager.cc:1905] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:43.236480 17334 ts_tablet_manager.cc:1918] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:43.236624 17334 log.cc:1196] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/5b2bea26c82e4464a643ca2f80a9e84d
I20260809 03:25:43.236939 17334 ts_tablet_manager.cc:1939] T 5b2bea26c82e4464a643ca2f80a9e84d P dfde0bfae5c34f62914fb7a50e962a74: Deleting consensus metadata
I20260809 03:25:43.237758 16511 catalog_manager.cc:4841] TS dfde0bfae5c34f62914fb7a50e962a74 (127.9.46.195:35363): tablet 5b2bea26c82e4464a643ca2f80a9e84d (table test-table [id=9ceca140cb4e4853b3921088be34750d]) successfully deleted
I20260809 03:25:43.240275 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 16920
I20260809 03:25:43.240361 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:43.240103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:34455 (local address 127.0.0.1:51200)
0809 03:25:43.240234 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:43.240285 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 03:25:43.241816 16514 catalog_manager.cc:4584] TS ffd2a2de7b574340948f0136a92d236e (127.9.46.193:34455): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b2bea26c82e4464a643ca2f80a9e84d: Network error: Client connection negotiation failed: client connection to 127.9.46.193:34455: connect: Connection refused (error 111)
I20260809 03:25:43.248778 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 17064
I20260809 03:25:43.256865 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 16384
I20260809 03:25:43.264307 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 16497
2026-08-09T03:25:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7311 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:43 dist-test-slave-65mx krb5kdc[17339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:43 dist-test-slave-65mx krb5kdc[17339](info): set up 2 sockets
Aug 09 03:25:43 dist-test-slave-65mx krb5kdc[17339](info): commencing operation
krb5kdc: starting...
W20260809 03:25:44.370265 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:44 dist-test-slave-65mx krb5kdc[17339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.254.keytab.
Entry for principal oryx/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.254.keytab.
I20260809 03:25:44.501636 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41909
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.254.keytab
--principal=oryx/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:41909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:44.613382 17353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:44.613720 17353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:44.613809 17353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:44.617914 17353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:44.618002 17353 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 03:25:44.618036 17353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:44.618079 17353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:44.618114 17353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:44.618170 17353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:44.623572 17353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:41909
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.254.keytab
--principal=oryx/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:44.624708 17353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:44.625730 17353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:44.634263 17363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:44.634262 17361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:44.634265 17360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:44.634652 17353 server_base.cc:1034] running on GCE node
I20260809 03:25:44.635257 17353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:44.636382 17353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:44.637547 17353 hybrid_clock.cc:648] HybridClock initialized: now 1786245944637530 us; error 41 us; skew 500 ppm
Aug 09 03:25:44 dist-test-slave-65mx krb5kdc[17339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245944, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:44.640357 17353 init.cc:377] Logged in from keytab as oryx/127.9.46.254@KRBTEST.COM (short username oryx)
I20260809 03:25:44.641302 17353 webserver.cc:460] Webserver started at http://127.9.46.254:34517/ using document root <none> and password file <none>
I20260809 03:25:44.641835 17353 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:44.641893 17353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:44.642066 17353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:44.644101 17353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "1c431a8aa3764054a31da9409773da22"
format_stamp: "Formatted at 2026-08-09 03:25:44 on dist-test-slave-65mx"
server_key: "8c22ca502233745cc651bdcd121fa91a"
server_key_iv: "2af04224010300243a841798fd04c64a"
server_key_version: "encryptionkey@0"
I20260809 03:25:44.644711 17353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "1c431a8aa3764054a31da9409773da22"
format_stamp: "Formatted at 2026-08-09 03:25:44 on dist-test-slave-65mx"
server_key: "8c22ca502233745cc651bdcd121fa91a"
server_key_iv: "2af04224010300243a841798fd04c64a"
server_key_version: "encryptionkey@0"
I20260809 03:25:44.648631 17353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 03:25:44.651053 17371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:44.652640 17353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 03:25:44.652812 17353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "1c431a8aa3764054a31da9409773da22"
format_stamp: "Formatted at 2026-08-09 03:25:44 on dist-test-slave-65mx"
server_key: "8c22ca502233745cc651bdcd121fa91a"
server_key_iv: "2af04224010300243a841798fd04c64a"
server_key_version: "encryptionkey@0"
I20260809 03:25:44.652944 17353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:44.694573 17353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:44.697642 17353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:44.697867 17353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:44.706194 17353 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:41909
I20260809 03:25:44.706225 17436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:41909 every 8 connection(s)
I20260809 03:25:44.707163 17353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:44.708972 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 17353
I20260809 03:25:44.709080 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:44.709375 9403 external_mini_cluster.cc:1428] Setting key a608e07a08195e76ec7b97e738358330
I20260809 03:25:44.710036 17437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:44.715842 17437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22: Bootstrap starting.
Aug 09 03:25:44 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:44.718756 17437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:44.719537 17437 log.cc:824] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:44.721549 17437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22: No bootstrap required, opened a new log
I20260809 03:25:44.724277 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:44.710785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55498 (local address 127.9.46.254:41909)
0809 03:25:44.711226 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:44.711241 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:44.711277 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:25:44.711927 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:44.711943 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:44.712257 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:44.712605 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:44.713429 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:44.714437 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:44.715276 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:44.715732 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:44.715896 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:44.719144 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:44.719149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:44.719161 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:44.719189 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:44.721432 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:44.721956 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:44.721968 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:44.721972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:44.722085 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:44.722382 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:44.722386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:44.722388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:44.722780 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:44.722999 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:44.723284 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 03:25:44.723584 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":145,"threads_started":1}
I20260809 03:25:44.724701 17437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c431a8aa3764054a31da9409773da22" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41909 } }
I20260809 03:25:44.724958 17437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:44.725051 17437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c431a8aa3764054a31da9409773da22, State: Initialized, Role: FOLLOWER
I20260809 03:25:44.725453 17437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c431a8aa3764054a31da9409773da22" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41909 } }
I20260809 03:25:44.725629 17437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:44.725713 17437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:44.725841 17437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:44.726799 17437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c431a8aa3764054a31da9409773da22" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41909 } }
I20260809 03:25:44.727140 17437 leader_election.cc:302] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c431a8aa3764054a31da9409773da22; no voters:
I20260809 03:25:44.727443 17437 leader_election.cc:288] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:44.727607 17442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:44.727845 17442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [term 1 LEADER]: Becoming Leader. State: Replica: 1c431a8aa3764054a31da9409773da22, State: Running, Role: LEADER
I20260809 03:25:44.728225 17442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c431a8aa3764054a31da9409773da22" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41909 } }
I20260809 03:25:44.728804 17437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:44.730067 17445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c431a8aa3764054a31da9409773da22. Latest consensus state: current_term: 1 leader_uuid: "1c431a8aa3764054a31da9409773da22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c431a8aa3764054a31da9409773da22" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41909 } } }
I20260809 03:25:44.730166 17445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:44.730130 17443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c431a8aa3764054a31da9409773da22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c431a8aa3764054a31da9409773da22" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41909 } } }
I20260809 03:25:44.730234 17443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22 [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:44.730511 17452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:44.733742 17452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:44.739303 17452 catalog_manager.cc:1347] Generated new cluster ID: b08ef395c3ef4030ab28505e34681875
I20260809 03:25:44.739389 17452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:44.766820 17452 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:44.767828 17452 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:44.772380 17452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22: Generated new TSK 0
I20260809 03:25:44.773125 17452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab.
Entry for principal oryx/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab.
I20260809 03:25:44.809749 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41909
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab
--principal=oryx/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:44.937698 17466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:44.938007 17466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:44.938067 17466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:44.942237 17466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:44.942301 17466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:44.942341 17466 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:25:44.942375 17466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:44.947845 17466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab
--principal=oryx/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.46.254:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:44.949026 17466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:44.950464 17466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:44.960605 17474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:44.960597 17476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:44.960781 17466 server_base.cc:1034] running on GCE node
W20260809 03:25:44.960613 17473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:44.961283 17466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:44.961988 17466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:44.963184 17466 hybrid_clock.cc:648] HybridClock initialized: now 1786245944963149 us; error 46 us; skew 500 ppm
Aug 09 03:25:44 dist-test-slave-65mx krb5kdc[17339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245944, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:44.966112 17466 init.cc:377] Logged in from keytab as oryx/127.9.46.193@KRBTEST.COM (short username oryx)
I20260809 03:25:44.967005 17466 webserver.cc:460] Webserver started at http://127.9.46.193:33397/ using document root <none> and password file <none>
I20260809 03:25:44.967633 17466 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:44.967732 17466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:44.967967 17466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:44.970108 17466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "5a824b778a724ec6b332a19a3097b423"
format_stamp: "Formatted at 2026-08-09 03:25:44 on dist-test-slave-65mx"
server_key: "29c5718f7bcf1ae8e0d05c18e160b5ab"
server_key_iv: "b536ff91638843962687993d62bd78ab"
server_key_version: "encryptionkey@0"
I20260809 03:25:44.970703 17466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "5a824b778a724ec6b332a19a3097b423"
format_stamp: "Formatted at 2026-08-09 03:25:44 on dist-test-slave-65mx"
server_key: "29c5718f7bcf1ae8e0d05c18e160b5ab"
server_key_iv: "b536ff91638843962687993d62bd78ab"
server_key_version: "encryptionkey@0"
I20260809 03:25:44.974677 17466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:25:44.977246 17483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:44.978466 17466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:44.978598 17466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "5a824b778a724ec6b332a19a3097b423"
format_stamp: "Formatted at 2026-08-09 03:25:44 on dist-test-slave-65mx"
server_key: "29c5718f7bcf1ae8e0d05c18e160b5ab"
server_key_iv: "b536ff91638843962687993d62bd78ab"
server_key_version: "encryptionkey@0"
I20260809 03:25:44.978713 17466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:45.007144 17466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:45.010541 17466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:45.010769 17466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:45.012259 17466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:45.012339 17466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:45.012416 17466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:45.012471 17466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:25:45 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245944, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:45.024216 17466 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:35993
I20260809 03:25:45.024251 17603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:35993 every 8 connection(s)
I20260809 03:25:45.025153 17466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:45.027828 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.013360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:42581 (local address 127.9.46.254:41909)
0809 03:25:45.013518 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.013522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.014600 (+ 1078us) server_negotiation.cc:407] Connection header received
0809 03:25:45.016086 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.016090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.016150 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.016239 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:45.017749 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.018374 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.020230 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.020723 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.020869 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.023736 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:45.023740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:45.023743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:45.023777 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:45.025565 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.026190 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.026196 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.026201 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.026301 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.026675 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.026678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.026680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.026892 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:45.027005 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.027502 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.027643 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 03:25:45.028466 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 17466
I20260809 03:25:45.028653 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:45.028684 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.013860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41909 (local address 127.9.46.193:42581)
0809 03:25:45.014367 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.014417 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.015827 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.016456 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.016466 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.016875 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:45.017592 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.017607 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.018508 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.018512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.020061 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.020073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.020923 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.020929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.021163 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.021787 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:45.021803 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:45.023527 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:45.025707 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.025713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.025727 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.026076 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.026412 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.026415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.026418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.026562 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.027019 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:45.027027 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:45.027266 (+ 239us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.027454 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.027769 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":133,"threads_started":1}
I20260809 03:25:45.029044 9403 external_mini_cluster.cc:1428] Setting key 03ef5ba551e530c2cafa7632cb4a9f81
I20260809 03:25:45.030105 17604 heartbeater.cc:347] Connected to a master server at 127.9.46.254:41909
I20260809 03:25:45.030256 17604 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:45.030877 17604 heartbeater.cc:511] Master 127.9.46.254:41909 requested a full tablet report, sending...
I20260809 03:25:45.032521 17387 ts_manager.cc:194] Registered new tserver with Master: 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993)
I20260809 03:25:45.034222 17387 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:42581
WARNING: no policy specified for oryx/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:25:45.040491 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.032650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:46339 (local address 127.9.46.254:41909)
0809 03:25:45.032797 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.032803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.033234 (+ 431us) server_negotiation.cc:407] Connection header received
0809 03:25:45.033351 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.033354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.033401 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.033487 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:45.034033 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.034748 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.035714 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.035945 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.036066 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.037324 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:45.037328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:45.037330 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:45.037354 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:45.038822 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.039264 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.039268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.039270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.039357 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.039655 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.039658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.039660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.039835 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:45.039941 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.040211 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.040321 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":45}
I20260809 03:25:45.040509 17611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.032583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41909 (local address 127.9.46.193:46339)
0809 03:25:45.033130 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.033144 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.033273 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.033506 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.033508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.033767 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:45.033892 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.033898 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.034904 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.034909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.035571 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.035581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.036073 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.036077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.036170 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.036798 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:45.036815 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:45.037183 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:45.038938 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.038941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.038944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.039155 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.039452 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.039455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.039457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.039553 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.039945 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:45.039949 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:45.040047 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.040155 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.040307 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":276,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab.
Entry for principal oryx/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab.
I20260809 03:25:45.060062 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41909
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab
--principal=oryx/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:45.175453 17613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:45.175729 17613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:45.175779 17613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:45.180344 17613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:45.180413 17613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:45.180454 17613 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:25:45.180486 17613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:45.186088 17613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab
--principal=oryx/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.46.254:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:45.187184 17613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:45.188174 17613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:45.197865 17620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:45.197911 17623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:45.197898 17621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:45.198184 17613 server_base.cc:1034] running on GCE node
I20260809 03:25:45.198730 17613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:45.199337 17613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:45.200579 17613 hybrid_clock.cc:648] HybridClock initialized: now 1786245945200507 us; error 51 us; skew 500 ppm
Aug 09 03:25:45 dist-test-slave-65mx krb5kdc[17339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245945, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:45.203423 17613 init.cc:377] Logged in from keytab as oryx/127.9.46.194@KRBTEST.COM (short username oryx)
I20260809 03:25:45.204277 17613 webserver.cc:460] Webserver started at http://127.9.46.194:44411/ using document root <none> and password file <none>
I20260809 03:25:45.204902 17613 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:45.204998 17613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:45.205242 17613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:45.207341 17613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "63d2ffdc2ce44df785eafca236753ac1"
format_stamp: "Formatted at 2026-08-09 03:25:45 on dist-test-slave-65mx"
server_key: "1efb758d163de6bd257af8c85ca8870c"
server_key_iv: "27388efab9b016b3f14d2a6547d7b83c"
server_key_version: "encryptionkey@0"
I20260809 03:25:45.207927 17613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "63d2ffdc2ce44df785eafca236753ac1"
format_stamp: "Formatted at 2026-08-09 03:25:45 on dist-test-slave-65mx"
server_key: "1efb758d163de6bd257af8c85ca8870c"
server_key_iv: "27388efab9b016b3f14d2a6547d7b83c"
server_key_version: "encryptionkey@0"
I20260809 03:25:45.211822 17613 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:25:45.214414 17630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:45.215740 17613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 03:25:45.215987 17613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "63d2ffdc2ce44df785eafca236753ac1"
format_stamp: "Formatted at 2026-08-09 03:25:45 on dist-test-slave-65mx"
server_key: "1efb758d163de6bd257af8c85ca8870c"
server_key_iv: "27388efab9b016b3f14d2a6547d7b83c"
server_key_version: "encryptionkey@0"
I20260809 03:25:45.216113 17613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:45.229148 17613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:45.232757 17613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:45.232970 17613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:45.234579 17613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:45.234655 17613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:45.234730 17613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:45.234774 17613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:25:45 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245945, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:45.245921 17613 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:39743
I20260809 03:25:45.245978 17757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:39743 every 8 connection(s)
I20260809 03:25:45.247095 17613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:45.247606 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 17613
I20260809 03:25:45.247738 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:45.248068 9403 external_mini_cluster.cc:1428] Setting key 34d15fa73c17cc970f50d2e27682ad26
I20260809 03:25:45.249966 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.235322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:51213 (local address 127.9.46.254:41909)
0809 03:25:45.235482 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.235489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.236348 (+ 859us) server_negotiation.cc:407] Connection header received
0809 03:25:45.237429 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.237435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.237512 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.237622 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:45.239470 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.240353 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.242310 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.242583 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.242704 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.245766 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:45.245770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:45.245772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:45.245803 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:45.247527 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.248196 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.248200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.248202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.248303 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.248776 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.248780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.248781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.249015 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:45.249145 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.249735 (+ 590us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.249846 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:25:45.251129 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.235707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41909 (local address 127.9.46.194:51213)
0809 03:25:45.236173 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.236225 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.237144 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.237974 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.237984 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.238572 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:45.239281 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.239306 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.240518 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.240523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.242161 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.242176 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.242761 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.242767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.243017 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.243675 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:45.243700 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:45.245533 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:45.247675 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.247680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.247697 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.248051 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.248429 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.248434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.248436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.248669 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.249173 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:45.249179 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:45.249474 (+ 295us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.249666 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.250009 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":134,"threads_started":1}
I20260809 03:25:45.252708 17758 heartbeater.cc:347] Connected to a master server at 127.9.46.254:41909
I20260809 03:25:45.252871 17758 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:45.253448 17758 heartbeater.cc:511] Master 127.9.46.254:41909 requested a full tablet report, sending...
I20260809 03:25:45.254618 17389 ts_manager.cc:194] Registered new tserver with Master: 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743)
I20260809 03:25:45.255407 17389 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:51213
WARNING: no policy specified for oryx/127.9.46.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.195@KRBTEST.COM" created.
I20260809 03:25:45.267867 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.254988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41909 (local address 127.9.46.194:57741)
0809 03:25:45.259882 (+ 4894us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.259897 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.260001 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.260259 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.260261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.260440 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:45.260633 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.260642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.261472 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.261474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.262140 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.262150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.262641 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.262643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.262752 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.263286 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:45.263304 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:45.263790 (+ 486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:45.266142 (+ 2352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.266144 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.266146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.266359 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.266706 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.266707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.266709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.266788 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.267385 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:45.267387 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:45.267470 (+ 83us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.267557 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.267672 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4802,"thread_start_us":105,"threads_started":1}
I20260809 03:25:45.268108 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.255086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:57741 (local address 127.9.46.254:41909)
0809 03:25:45.255232 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.255239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.259998 (+ 4759us) server_negotiation.cc:407] Connection header received
0809 03:25:45.260087 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.260094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.260143 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.260218 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:45.260769 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.261340 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.262247 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.262501 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.262627 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.264017 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:45.264021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:45.264023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:45.264050 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:45.266003 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.266484 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.266491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.266495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.266601 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.266898 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.266905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.266910 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.267273 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:45.267395 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.267655 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.267818 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab.
Entry for principal oryx/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab.
I20260809 03:25:45.281895 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41909
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab
--principal=oryx/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:45.396624 17768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:45.396890 17768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:45.396982 17768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:45.401105 17768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:45.401162 17768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:45.401201 17768 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 03:25:45.401296 17768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:45.406883 17768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab
--principal=oryx/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.46.254:41909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:45.408336 17768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:45.409806 17768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:45.421355 17774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:45.421317 17776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:45.421324 17773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:45.421550 17768 server_base.cc:1034] running on GCE node
I20260809 03:25:45.421947 17768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:45.422551 17768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:45.423745 17768 hybrid_clock.cc:648] HybridClock initialized: now 1786245945423715 us; error 42 us; skew 500 ppm
Aug 09 03:25:45 dist-test-slave-65mx krb5kdc[17339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245945, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:45.426592 17768 init.cc:377] Logged in from keytab as oryx/127.9.46.195@KRBTEST.COM (short username oryx)
I20260809 03:25:45.427444 17768 webserver.cc:460] Webserver started at http://127.9.46.195:39715/ using document root <none> and password file <none>
I20260809 03:25:45.428013 17768 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:45.428105 17768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:45.428421 17768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:45.430755 17768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "928cac1251304933a300d17de83c3229"
format_stamp: "Formatted at 2026-08-09 03:25:45 on dist-test-slave-65mx"
server_key: "32a64f385244077fd208e244cb23628b"
server_key_iv: "503c72926e421670c3b224f9d1b4f3c8"
server_key_version: "encryptionkey@0"
I20260809 03:25:45.431450 17768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "928cac1251304933a300d17de83c3229"
format_stamp: "Formatted at 2026-08-09 03:25:45 on dist-test-slave-65mx"
server_key: "32a64f385244077fd208e244cb23628b"
server_key_iv: "503c72926e421670c3b224f9d1b4f3c8"
server_key_version: "encryptionkey@0"
I20260809 03:25:45.435370 17768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:25:45.437943 17783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:45.439158 17768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:25:45.439299 17768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "928cac1251304933a300d17de83c3229"
format_stamp: "Formatted at 2026-08-09 03:25:45 on dist-test-slave-65mx"
server_key: "32a64f385244077fd208e244cb23628b"
server_key_iv: "503c72926e421670c3b224f9d1b4f3c8"
server_key_version: "encryptionkey@0"
I20260809 03:25:45.439415 17768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:45.457788 17768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:45.461199 17768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:45.461408 17768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:45.462847 17768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:45.462919 17768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:45.462996 17768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:45.463043 17768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 03:25:45 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245945, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:45.474236 17768 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:39993
I20260809 03:25:45.475062 17768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:45.475864 17903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:39993 every 8 connection(s)
I20260809 03:25:45.477393 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.463835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:60081 (local address 127.9.46.254:41909)
0809 03:25:45.464014 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.464019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.465080 (+ 1061us) server_negotiation.cc:407] Connection header received
0809 03:25:45.466197 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.466200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.466267 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.466346 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:45.468169 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.469101 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.470520 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.470767 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.470886 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.473646 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:45.473650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:45.473653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:45.473680 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:45.475280 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.475903 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.475906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.475908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.476030 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.476362 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.476364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.476365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.476584 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:45.476689 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.477175 (+ 486us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.477277 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:25:45.478210 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.464387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41909 (local address 127.9.46.195:60081)
0809 03:25:45.464911 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.464954 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.465926 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.466822 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.466830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.467256 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:45.467956 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.467973 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.469264 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.469269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.470381 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.470391 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.470938 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.470942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.471127 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.471732 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:45.471747 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:45.473478 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:45.475394 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.475399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.475412 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.475771 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.476143 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.476146 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.476147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.476275 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.476703 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:45.476709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:45.476930 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.477115 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.477357 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":160,"threads_started":1}
I20260809 03:25:45.479485 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 17768
I20260809 03:25:45.479602 17904 heartbeater.cc:347] Connected to a master server at 127.9.46.254:41909
I20260809 03:25:45.479678 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:45.479743 17904 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:45.479974 9403 external_mini_cluster.cc:1428] Setting key 188c6512786e2d55f822c86ee10948a1
I20260809 03:25:45.480274 17904 heartbeater.cc:511] Master 127.9.46.254:41909 requested a full tablet report, sending...
I20260809 03:25:45.481360 17389 ts_manager.cc:194] Registered new tserver with Master: 928cac1251304933a300d17de83c3229 (127.9.46.195:39993)
I20260809 03:25:45.482020 17389 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:60081
I20260809 03:25:45.482306 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:25:45.490221 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.481972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:43055 (local address 127.9.46.254:41909)
0809 03:25:45.482094 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.482098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.482360 (+ 262us) server_negotiation.cc:407] Connection header received
0809 03:25:45.482573 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.482578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.482619 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.482701 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:45.483215 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.483934 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.485151 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.485359 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.485444 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.486610 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:45.486616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:45.486622 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:45.486654 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:45.488304 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.488956 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.488960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.488962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.489048 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.489388 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.489390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.489391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.489556 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:45.489627 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.489994 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.490102 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:25:45.490376 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.481911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41909 (local address 127.9.46.195:43055)
0809 03:25:45.482261 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.482276 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.482398 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.482721 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.482723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.482941 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:45.483078 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.483085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.484110 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.484113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.485015 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.485029 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.485479 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.485483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.485586 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.486145 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:45.486157 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:45.486472 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:45.488433 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.488436 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.488437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.488758 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.489167 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:45.489174 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:45.489178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:45.489288 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:45.489655 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:45.489660 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:45.489784 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.489910 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.490083 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":114,"threads_started":1}
Aug 09 03:25:45 dist-test-slave-65mx krb5kdc[17339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245945, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 03:25:45 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:45 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:25:45.509800 17440 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:45.503728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43490 (local address 127.9.46.254:41909)
0809 03:25:45.503869 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.503873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.503885 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:25:45.503927 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.503929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.503971 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.504047 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:45.504804 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.505325 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.506168 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.506375 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.506453 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.509654 (+ 3201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43490: BlockingRecv error: recv got EOF from 127.0.0.1:43490 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 03:25:45 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245945, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:45.518956 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.511187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.9.46.254:41909)
0809 03:25:45.511337 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.511341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.511473 (+ 132us) server_negotiation.cc:407] Connection header received
0809 03:25:45.511607 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.511611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.511659 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.511742 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:45.512317 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.512871 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.513650 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.513860 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.513971 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.516124 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:45.516127 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:45.516129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:45.516152 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:45.517631 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.517982 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.517985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.517986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.518059 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:45.518303 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:45.518305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:45.518306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:45.518465 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:45.518562 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.518749 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.518843 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 03:25:45.522051 17389 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:45.525089 17389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:45.539126 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.533575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51624 (local address 127.9.46.194:39743)
0809 03:25:45.534094 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.534110 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.534138 (+ 28us) server_negotiation.cc:407] Connection header received
0809 03:25:45.534226 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.534231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.534431 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.534597 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:45.534890 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.536101 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.537611 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.538328 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.538459 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.538636 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.538878 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.538974 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":133,"threads_started":1}
I20260809 03:25:45.539129 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.533410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:39743 (local address 127.0.0.1:51624)
0809 03:25:45.533753 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.533803 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.533934 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.534594 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.534602 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.534629 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:45.534761 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.534769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.536219 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.536223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.537453 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.537464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.538536 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.538545 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.538664 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.538706 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.538835 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.538948 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":21,"thread_start_us":91,"threads_started":1}
I20260809 03:25:45.540045 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.534094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:39993 (local address 127.0.0.1:34590)
0809 03:25:45.534919 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.534935 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.535023 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.535511 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.535513 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.535532 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:45.535618 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.535627 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.536976 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.536979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.538272 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.538283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.539632 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.539636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.539742 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.539765 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.539813 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.539883 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":51,"thread_start_us":101,"threads_started":1}
I20260809 03:25:45.540056 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.534495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.9.46.193:35993)
0809 03:25:45.535263 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.535269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.535290 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:25:45.535359 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.535363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.535541 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.535715 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:45.535933 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.536806 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.538560 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.539359 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.539525 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.539661 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.539822 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.539909 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":93,"threads_started":1}
I20260809 03:25:45.540078 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.534033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:35993 (local address 127.0.0.1:35654)
0809 03:25:45.534305 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.534319 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.534411 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.535699 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.535704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.535725 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:45.535809 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.535817 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.536918 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.536924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.538366 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.538374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.539537 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.539541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.539628 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.539690 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.539791 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.539864 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":125,"thread_start_us":132,"threads_started":1}
I20260809 03:25:45.540917 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.534527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34590 (local address 127.9.46.195:39993)
0809 03:25:45.535049 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.535061 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.535084 (+ 23us) server_negotiation.cc:407] Connection header received
0809 03:25:45.535160 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.535168 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.535351 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.535522 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:45.535744 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.536820 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.538410 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.539165 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.540455 (+ 1290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.540649 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.540700 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.540784 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":102,"threads_started":1}
I20260809 03:25:45.542232 17685 tablet_service.cc:1467] Processing CreateTablet for tablet 927fa2bd4a924c268138edafd86042ab (DEFAULT_TABLE table=test-table [id=89c10ecd7b7b449e93393e4ca1bcb691]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:45.543419 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927fa2bd4a924c268138edafd86042ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:45.543303 17535 tablet_service.cc:1467] Processing CreateTablet for tablet 927fa2bd4a924c268138edafd86042ab (DEFAULT_TABLE table=test-table [id=89c10ecd7b7b449e93393e4ca1bcb691]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:45.543597 17835 tablet_service.cc:1467] Processing CreateTablet for tablet 927fa2bd4a924c268138edafd86042ab (DEFAULT_TABLE table=test-table [id=89c10ecd7b7b449e93393e4ca1bcb691]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:45.544292 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927fa2bd4a924c268138edafd86042ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:45.544512 17835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927fa2bd4a924c268138edafd86042ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:45.550625 17934 tablet_bootstrap.cc:492] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: Bootstrap starting.
I20260809 03:25:45.552433 17935 tablet_bootstrap.cc:492] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: Bootstrap starting.
I20260809 03:25:45.553169 17934 tablet_bootstrap.cc:654] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:45.554028 17934 log.cc:824] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:45.554260 17935 tablet_bootstrap.cc:654] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:45.554909 17935 log.cc:824] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:45.555572 17936 tablet_bootstrap.cc:492] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: Bootstrap starting.
I20260809 03:25:45.556978 17935 tablet_bootstrap.cc:492] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: No bootstrap required, opened a new log
I20260809 03:25:45.557178 17935 ts_tablet_manager.cc:1397] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:25:45.557606 17936 tablet_bootstrap.cc:654] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:45.558260 17936 log.cc:824] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:45.560324 17934 tablet_bootstrap.cc:492] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: No bootstrap required, opened a new log
I20260809 03:25:45.560518 17934 ts_tablet_manager.cc:1397] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260809 03:25:45.560699 17935 raft_consensus.cc:348] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } }
I20260809 03:25:45.561009 17935 raft_consensus.cc:374] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:45.561076 17935 raft_consensus.cc:729] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d2ffdc2ce44df785eafca236753ac1, State: Initialized, Role: FOLLOWER
I20260809 03:25:45.561563 17935 consensus_queue.cc:260] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } }
I20260809 03:25:45.561713 17936 tablet_bootstrap.cc:492] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: No bootstrap required, opened a new log
I20260809 03:25:45.561885 17936 ts_tablet_manager.cc:1397] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 03:25:45.562590 17935 ts_tablet_manager.cc:1428] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:25:45.562935 17758 heartbeater.cc:503] Master 127.9.46.254:41909 was elected leader, sending a full tablet report...
I20260809 03:25:45.564766 17934 raft_consensus.cc:348] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } }
I20260809 03:25:45.564996 17934 raft_consensus.cc:374] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:45.565060 17934 raft_consensus.cc:729] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a824b778a724ec6b332a19a3097b423, State: Initialized, Role: FOLLOWER
I20260809 03:25:45.565485 17934 consensus_queue.cc:260] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } }
I20260809 03:25:45.565390 17936 raft_consensus.cc:348] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } }
I20260809 03:25:45.565652 17936 raft_consensus.cc:374] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:45.565714 17936 raft_consensus.cc:729] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928cac1251304933a300d17de83c3229, State: Initialized, Role: FOLLOWER
I20260809 03:25:45.566119 17936 consensus_queue.cc:260] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } }
I20260809 03:25:45.566246 17604 heartbeater.cc:503] Master 127.9.46.254:41909 was elected leader, sending a full tablet report...
I20260809 03:25:45.566440 17934 ts_tablet_manager.cc:1428] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260809 03:25:45.566982 17936 ts_tablet_manager.cc:1428] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:25:45.567305 17904 heartbeater.cc:503] Master 127.9.46.254:41909 was elected leader, sending a full tablet report...
I20260809 03:25:45.617427 17941 raft_consensus.cc:482] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:45.617660 17941 raft_consensus.cc:504] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } }
I20260809 03:25:45.618774 17941 leader_election.cc:288] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743), 928cac1251304933a300d17de83c3229 (127.9.46.195:39993)
I20260809 03:25:45.623306 17945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.619684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:39993 (local address 127.9.46.193:42059)
0809 03:25:45.620022 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.620036 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.620123 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.620399 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.620402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.620422 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:45.620536 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.620573 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.621299 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.621302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.622195 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.622204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.622856 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.622859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.622950 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.622982 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.623054 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.623109 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":34,"thread_start_us":108,"threads_started":1}
I20260809 03:25:45.623311 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.619830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:42059 (local address 127.9.46.195:39993)
0809 03:25:45.619984 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.619989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.620109 (+ 120us) server_negotiation.cc:407] Connection header received
0809 03:25:45.620233 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.620239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.620289 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.620375 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:45.620662 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.621179 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.622327 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.622730 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.622848 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.622909 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.623096 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.623184 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 03:25:45.623380 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.619063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:39743 (local address 127.9.46.193:57049)
0809 03:25:45.619535 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.619548 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.619753 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.620100 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.620104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.620141 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:45.620281 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.620288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.621231 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.621237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.622209 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.622219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.622908 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.622911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.623059 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.623086 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.623164 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.623228 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":63,"spinlock_wait_cycles":896,"thread_start_us":216,"threads_started":1}
I20260809 03:25:45.623409 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.619195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:57049 (local address 127.9.46.194:39743)
0809 03:25:45.619383 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.619388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.619678 (+ 290us) server_negotiation.cc:407] Connection header received
0809 03:25:45.619892 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.619895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.619955 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.620038 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:45.620390 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.621112 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.622369 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.622755 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.622894 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.622948 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.623203 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.623277 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 03:25:45.624033 17706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927fa2bd4a924c268138edafd86042ab" candidate_uuid: "5a824b778a724ec6b332a19a3097b423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d2ffdc2ce44df785eafca236753ac1" is_pre_election: true
I20260809 03:25:45.624123 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927fa2bd4a924c268138edafd86042ab" candidate_uuid: "5a824b778a724ec6b332a19a3097b423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928cac1251304933a300d17de83c3229" is_pre_election: true
I20260809 03:25:45.624392 17856 raft_consensus.cc:2454] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a824b778a724ec6b332a19a3097b423 in term 0.
I20260809 03:25:45.624393 17706 raft_consensus.cc:2454] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a824b778a724ec6b332a19a3097b423 in term 0.
I20260809 03:25:45.624933 17485 leader_election.cc:302] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a824b778a724ec6b332a19a3097b423, 928cac1251304933a300d17de83c3229; no voters:
I20260809 03:25:45.625195 17941 raft_consensus.cc:2781] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:45.625281 17941 raft_consensus.cc:482] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:45.625324 17941 raft_consensus.cc:3037] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:45.626406 17941 raft_consensus.cc:504] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } }
I20260809 03:25:45.626751 17941 leader_election.cc:288] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [CANDIDATE]: Term 1 election: Requested vote from peers 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743), 928cac1251304933a300d17de83c3229 (127.9.46.195:39993)
I20260809 03:25:45.627108 17706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927fa2bd4a924c268138edafd86042ab" candidate_uuid: "5a824b778a724ec6b332a19a3097b423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d2ffdc2ce44df785eafca236753ac1"
I20260809 03:25:45.627157 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927fa2bd4a924c268138edafd86042ab" candidate_uuid: "5a824b778a724ec6b332a19a3097b423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928cac1251304933a300d17de83c3229"
I20260809 03:25:45.627249 17706 raft_consensus.cc:3037] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:45.627266 17856 raft_consensus.cc:3037] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:45.629055 17706 raft_consensus.cc:2454] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a824b778a724ec6b332a19a3097b423 in term 1.
I20260809 03:25:45.629036 17856 raft_consensus.cc:2454] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a824b778a724ec6b332a19a3097b423 in term 1.
I20260809 03:25:45.629386 17485 leader_election.cc:302] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a824b778a724ec6b332a19a3097b423, 63d2ffdc2ce44df785eafca236753ac1; no voters:
I20260809 03:25:45.629774 17941 raft_consensus.cc:2781] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:45.630033 17941 raft_consensus.cc:686] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 1 LEADER]: Becoming Leader. State: Replica: 5a824b778a724ec6b332a19a3097b423, State: Running, Role: LEADER
I20260809 03:25:45.630373 17941 consensus_queue.cc:237] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } }
I20260809 03:25:45.633797 17389 catalog_manager.cc:5495] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a824b778a724ec6b332a19a3097b423 (127.9.46.193). New cstate: current_term: 1 leader_uuid: "5a824b778a724ec6b332a19a3097b423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } health_report { overall_health: HEALTHY } } }
I20260809 03:25:45.662247 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.658808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35670 (local address 127.9.46.193:35993)
0809 03:25:45.658956 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.658960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.658974 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:25:45.659065 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.659068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.659132 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.659230 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:45.659482 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.660035 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.660918 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.661151 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.661235 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.661475 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:45.661920 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:45.662013 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.662076 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.662129 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:25:45.668097 17856 raft_consensus.cc:1261] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Refusing update from remote peer 5a824b778a724ec6b332a19a3097b423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:45.668166 17706 raft_consensus.cc:1261] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Refusing update from remote peer 5a824b778a724ec6b332a19a3097b423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:45.668848 17941 consensus_queue.cc:1035] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:45.669085 17946 consensus_queue.cc:1035] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:45.688351 17389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:45.688585 17389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:45.691141 17389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22: Sending DeleteTablet for 3 replicas of tablet 927fa2bd4a924c268138edafd86042ab
I20260809 03:25:45.691984 17535 tablet_service.cc:1514] Processing DeleteTablet for tablet 927fa2bd4a924c268138edafd86042ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:45 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:35654
I20260809 03:25:45.692287 17835 tablet_service.cc:1514] Processing DeleteTablet for tablet 927fa2bd4a924c268138edafd86042ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:45 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:34590
I20260809 03:25:45.692363 17685 tablet_service.cc:1514] Processing DeleteTablet for tablet 927fa2bd4a924c268138edafd86042ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:45 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:51624
I20260809 03:25:45.692898 17962 tablet_replica.cc:330] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: stopping tablet replica
I20260809 03:25:45.693006 17963 tablet_replica.cc:330] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: stopping tablet replica
I20260809 03:25:45.693205 17387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:25:45.693354 17963 raft_consensus.cc:2229] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:45.693545 17962 raft_consensus.cc:2229] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 03:25:45.693645 17387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:45.693713 17963 raft_consensus.cc:2258] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:45.694186 17962 raft_consensus.cc:2258] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:45.695577 17963 ts_tablet_manager.cc:1905] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:45.695760 17962 ts_tablet_manager.cc:1905] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:45.698582 17963 ts_tablet_manager.cc:1918] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:45.698704 17963 log.cc:1196] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/927fa2bd4a924c268138edafd86042ab
I20260809 03:25:45.698719 17962 ts_tablet_manager.cc:1918] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:45.698957 17962 log.cc:1196] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/927fa2bd4a924c268138edafd86042ab
I20260809 03:25:45.699438 17535 tablet_service.cc:1467] Processing CreateTablet for tablet acdf549c48a1415baab48278b0085bd9 (DEFAULT_TABLE table=test-table [id=c5ca89ce69924f2bbb230630e68d08f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:45.699486 17835 tablet_service.cc:1467] Processing CreateTablet for tablet acdf549c48a1415baab48278b0085bd9 (DEFAULT_TABLE table=test-table [id=c5ca89ce69924f2bbb230630e68d08f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:45.699739 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdf549c48a1415baab48278b0085bd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:45.699856 17835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdf549c48a1415baab48278b0085bd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:45.700363 17961 tablet_replica.cc:330] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: stopping tablet replica
I20260809 03:25:45.700682 17961 raft_consensus.cc:2229] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:45.701324 17961 raft_consensus.cc:2258] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:45.702169 17936 tablet_bootstrap.cc:492] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: Bootstrap starting.
I20260809 03:25:45.702320 17934 tablet_bootstrap.cc:492] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: Bootstrap starting.
I20260809 03:25:45.702736 17963 ts_tablet_manager.cc:1939] T 927fa2bd4a924c268138edafd86042ab P 928cac1251304933a300d17de83c3229: Deleting consensus metadata
I20260809 03:25:45.699436 17685 tablet_service.cc:1467] Processing CreateTablet for tablet acdf549c48a1415baab48278b0085bd9 (DEFAULT_TABLE table=test-table [id=c5ca89ce69924f2bbb230630e68d08f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:45.703006 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdf549c48a1415baab48278b0085bd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:45.703218 17936 tablet_bootstrap.cc:654] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:45.703439 17934 tablet_bootstrap.cc:654] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:45.704109 17373 catalog_manager.cc:4841] TS 928cac1251304933a300d17de83c3229 (127.9.46.195:39993): tablet 927fa2bd4a924c268138edafd86042ab (table test-table [id=89c10ecd7b7b449e93393e4ca1bcb691]) successfully deleted
I20260809 03:25:45.705272 17961 ts_tablet_manager.cc:1905] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:45.705327 17934 tablet_bootstrap.cc:492] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: No bootstrap required, opened a new log
I20260809 03:25:45.705410 17934 ts_tablet_manager.cc:1397] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 03:25:45.705420 17935 tablet_bootstrap.cc:492] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: Bootstrap starting.
I20260809 03:25:45.705912 17934 raft_consensus.cc:348] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:45.706063 17934 raft_consensus.cc:374] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:45.706104 17934 raft_consensus.cc:729] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a824b778a724ec6b332a19a3097b423, State: Initialized, Role: FOLLOWER
I20260809 03:25:45.706233 17934 consensus_queue.cc:260] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:45.706501 17934 ts_tablet_manager.cc:1428] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260809 03:25:45.707487 17962 ts_tablet_manager.cc:1939] T 927fa2bd4a924c268138edafd86042ab P 63d2ffdc2ce44df785eafca236753ac1: Deleting consensus metadata
I20260809 03:25:45.708789 17373 catalog_manager.cc:4841] TS 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743): tablet 927fa2bd4a924c268138edafd86042ab (table test-table [id=89c10ecd7b7b449e93393e4ca1bcb691]) successfully deleted
I20260809 03:25:45.709062 17935 tablet_bootstrap.cc:654] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:45.711462 17961 ts_tablet_manager.cc:1918] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:45.711575 17961 log.cc:1196] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/927fa2bd4a924c268138edafd86042ab
I20260809 03:25:45.711879 17961 ts_tablet_manager.cc:1939] T 927fa2bd4a924c268138edafd86042ab P 5a824b778a724ec6b332a19a3097b423: Deleting consensus metadata
I20260809 03:25:45.713368 17936 tablet_bootstrap.cc:492] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: No bootstrap required, opened a new log
I20260809 03:25:45.713472 17936 ts_tablet_manager.cc:1397] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260809 03:25:45.713451 17374 catalog_manager.cc:4841] TS 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993): tablet 927fa2bd4a924c268138edafd86042ab (table test-table [id=89c10ecd7b7b449e93393e4ca1bcb691]) successfully deleted
I20260809 03:25:45.713966 17936 raft_consensus.cc:348] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:45.714071 17936 raft_consensus.cc:374] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:45.714095 17936 raft_consensus.cc:729] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928cac1251304933a300d17de83c3229, State: Initialized, Role: FOLLOWER
I20260809 03:25:45.714243 17936 consensus_queue.cc:260] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:45.714495 17936 ts_tablet_manager.cc:1428] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260809 03:25:45.726116 17905 tablet.cc:2369] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:45.726635 17935 tablet_bootstrap.cc:492] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: No bootstrap required, opened a new log
I20260809 03:25:45.726753 17935 ts_tablet_manager.cc:1397] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20260809 03:25:45.727289 17935 raft_consensus.cc:348] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:45.727432 17935 raft_consensus.cc:374] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:45.727466 17935 raft_consensus.cc:729] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d2ffdc2ce44df785eafca236753ac1, State: Initialized, Role: FOLLOWER
I20260809 03:25:45.727619 17935 consensus_queue.cc:260] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:45.727909 17935 ts_tablet_manager.cc:1428] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260809 03:25:45.748522 17759 tablet.cc:2369] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:25:45.776779 17605 tablet.cc:2369] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:25:45.929214 17943 raft_consensus.cc:482] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:45.929414 17943 raft_consensus.cc:504] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:45.930806 17943 leader_election.cc:288] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993), 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743)
I20260809 03:25:45.934808 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.931199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:58639 (local address 127.9.46.194:39743)
0809 03:25:45.931573 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.931580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.931597 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:25:45.931659 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.931661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.931698 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.931786 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:45.932109 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.932696 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.933799 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.934212 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.934344 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.934400 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.934613 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.934684 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270}
I20260809 03:25:45.934808 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.931118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:39743 (local address 127.9.46.195:58639)
0809 03:25:45.931345 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.931362 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.931502 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.931816 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.931819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.931834 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:45.931976 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.931982 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.932813 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.932817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.933661 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.933671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.934338 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.934342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.934484 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.934502 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.934596 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.934674 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":43}
I20260809 03:25:45.934906 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.931267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:47009 (local address 127.9.46.193:35993)
0809 03:25:45.931434 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:45.931438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:45.931630 (+ 192us) server_negotiation.cc:407] Connection header received
0809 03:25:45.931764 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:45.931769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:45.931818 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:45.931940 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:45.932238 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.932823 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.933885 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.934286 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.934423 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.934480 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:45.934698 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 03:25:45.934777 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:25:45.934971 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:45.931143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:35993 (local address 127.9.46.195:47009)
0809 03:25:45.931505 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:45.931518 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:45.931663 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:45.931964 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:45.931966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:45.931983 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:45.932119 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.932125 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.932934 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.932939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.933745 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:45.933754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:45.934423 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:45.934426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:45.934561 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:45.934589 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 03:25:45.934680 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 03:25:45.934749 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
I20260809 03:25:45.935418 17706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdf549c48a1415baab48278b0085bd9" candidate_uuid: "928cac1251304933a300d17de83c3229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d2ffdc2ce44df785eafca236753ac1" is_pre_election: true
I20260809 03:25:45.935499 17556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdf549c48a1415baab48278b0085bd9" candidate_uuid: "928cac1251304933a300d17de83c3229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a824b778a724ec6b332a19a3097b423" is_pre_election: true
I20260809 03:25:45.935591 17706 raft_consensus.cc:2454] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 928cac1251304933a300d17de83c3229 in term 0.
I20260809 03:25:45.935781 17556 raft_consensus.cc:2454] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 928cac1251304933a300d17de83c3229 in term 0.
I20260809 03:25:45.935963 17785 leader_election.cc:302] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63d2ffdc2ce44df785eafca236753ac1, 928cac1251304933a300d17de83c3229; no voters:
I20260809 03:25:45.936208 17943 raft_consensus.cc:2781] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:45.936267 17943 raft_consensus.cc:482] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:45.936311 17943 raft_consensus.cc:3037] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:45.937606 17943 raft_consensus.cc:504] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:45.938032 17943 leader_election.cc:288] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [CANDIDATE]: Term 1 election: Requested vote from peers 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993), 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743)
I20260809 03:25:45.938382 17556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdf549c48a1415baab48278b0085bd9" candidate_uuid: "928cac1251304933a300d17de83c3229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a824b778a724ec6b332a19a3097b423"
I20260809 03:25:45.938441 17706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdf549c48a1415baab48278b0085bd9" candidate_uuid: "928cac1251304933a300d17de83c3229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d2ffdc2ce44df785eafca236753ac1"
I20260809 03:25:45.938532 17556 raft_consensus.cc:3037] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:45.938604 17706 raft_consensus.cc:3037] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:45.939525 17556 raft_consensus.cc:2454] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928cac1251304933a300d17de83c3229 in term 1.
I20260809 03:25:45.939553 17706 raft_consensus.cc:2454] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928cac1251304933a300d17de83c3229 in term 1.
I20260809 03:25:45.939841 17786 leader_election.cc:302] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a824b778a724ec6b332a19a3097b423, 928cac1251304933a300d17de83c3229; no voters:
I20260809 03:25:45.940024 17943 raft_consensus.cc:2781] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:45.940208 17943 raft_consensus.cc:686] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 1 LEADER]: Becoming Leader. State: Replica: 928cac1251304933a300d17de83c3229, State: Running, Role: LEADER
I20260809 03:25:45.940464 17943 consensus_queue.cc:237] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:45.942840 17389 catalog_manager.cc:5495] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 reported cstate change: term changed from 0 to 1, leader changed from <none> to 928cac1251304933a300d17de83c3229 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "928cac1251304933a300d17de83c3229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } health_report { overall_health: HEALTHY } } }
Aug 09 03:25:46 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245944, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.254@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:46.027304 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.018386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41909 (local address 127.0.0.1:43512)
0809 03:25:46.018685 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:46.018697 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:46.018836 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:46.019123 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:46.019125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:46.019533 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:46.019665 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.019677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.020581 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.020584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.021207 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.021216 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.021610 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.021612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.021672 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.022212 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:46.022224 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:46.023668 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:46.025603 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:46.025612 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:46.025617 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:46.025886 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:46.026238 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:46.026240 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:46.026242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:46.026334 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:46.026763 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:46.026768 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:46.026877 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 03:25:46.027018 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 03:25:46.027158 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":96,"threads_started":1}
I20260809 03:25:46.027411 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.018444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43512 (local address 127.9.46.254:41909)
0809 03:25:46.018588 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.018594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.018774 (+ 180us) server_negotiation.cc:407] Connection header received
0809 03:25:46.018946 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.018949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.018985 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.019073 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:46.019788 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.020453 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.021310 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.021512 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.021594 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.023791 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:46.023794 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:46.023796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:46.023816 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:46.025461 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:46.026039 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:46.026042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:46.026044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:46.026128 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:46.026433 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:46.026436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:46.026438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:46.026654 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:46.026746 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.027076 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.027234 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":51}
I20260809 03:25:46.032155 17389 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:43512:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 03:25:46.033003 17389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:46.037914 17535 tablet_service.cc:1467] Processing CreateTablet for tablet 9908ba4235fa49c9b6aae373bf079de0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9fa6aca3b19f4e3a89dba35d723efc2c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:25:46.038085 17835 tablet_service.cc:1467] Processing CreateTablet for tablet 9908ba4235fa49c9b6aae373bf079de0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9fa6aca3b19f4e3a89dba35d723efc2c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:25:46.038156 17685 tablet_service.cc:1467] Processing CreateTablet for tablet 9908ba4235fa49c9b6aae373bf079de0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9fa6aca3b19f4e3a89dba35d723efc2c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 03:25:46.038343 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9908ba4235fa49c9b6aae373bf079de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:46.038424 17835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9908ba4235fa49c9b6aae373bf079de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:46.038564 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9908ba4235fa49c9b6aae373bf079de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:46.041338 17935 tablet_bootstrap.cc:492] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: Bootstrap starting.
I20260809 03:25:46.041394 17934 tablet_bootstrap.cc:492] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: Bootstrap starting.
I20260809 03:25:46.042158 17934 tablet_bootstrap.cc:654] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:46.042254 17935 tablet_bootstrap.cc:654] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:46.042937 17936 tablet_bootstrap.cc:492] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: Bootstrap starting.
I20260809 03:25:46.043604 17935 tablet_bootstrap.cc:492] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: No bootstrap required, opened a new log
I20260809 03:25:46.043672 17935 ts_tablet_manager.cc:1397] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:46.043823 17936 tablet_bootstrap.cc:654] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:46.044076 17935 raft_consensus.cc:348] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.044184 17935 raft_consensus.cc:374] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:46.044214 17935 raft_consensus.cc:729] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d2ffdc2ce44df785eafca236753ac1, State: Initialized, Role: FOLLOWER
I20260809 03:25:46.044329 17935 consensus_queue.cc:260] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.044631 17935 ts_tablet_manager.cc:1428] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:46.044584 17940 tablet_replica.cc:439] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } }
I20260809 03:25:46.044719 17940 tablet_replica.cc:442] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:46.044929 17934 tablet_bootstrap.cc:492] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: No bootstrap required, opened a new log
I20260809 03:25:46.045032 17934 ts_tablet_manager.cc:1397] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260809 03:25:46.045496 17934 raft_consensus.cc:348] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.045626 17934 raft_consensus.cc:374] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:46.045672 17934 raft_consensus.cc:729] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a824b778a724ec6b332a19a3097b423, State: Initialized, Role: FOLLOWER
I20260809 03:25:46.045797 17934 consensus_queue.cc:260] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.046105 17934 ts_tablet_manager.cc:1428] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:46.046674 17936 tablet_bootstrap.cc:492] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: No bootstrap required, opened a new log
I20260809 03:25:46.046748 17936 ts_tablet_manager.cc:1397] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 03:25:46.047086 17936 raft_consensus.cc:348] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.047178 17936 raft_consensus.cc:374] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:46.047200 17936 raft_consensus.cc:729] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928cac1251304933a300d17de83c3229, State: Initialized, Role: FOLLOWER
I20260809 03:25:46.047328 17936 consensus_queue.cc:260] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.047621 17936 ts_tablet_manager.cc:1428] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:25:46.047565 17943 tablet_replica.cc:439] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } }
I20260809 03:25:46.047695 17943 tablet_replica.cc:442] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:46.048233 17946 tablet_replica.cc:439] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } }
I20260809 03:25:46.048380 17946 tablet_replica.cc:442] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:46.066668 17943 raft_consensus.cc:482] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:46.066800 17943 raft_consensus.cc:504] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.067245 17943 leader_election.cc:288] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993), 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743)
I20260809 03:25:46.067691 17556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9908ba4235fa49c9b6aae373bf079de0" candidate_uuid: "928cac1251304933a300d17de83c3229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a824b778a724ec6b332a19a3097b423" is_pre_election: true
I20260809 03:25:46.067695 17706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9908ba4235fa49c9b6aae373bf079de0" candidate_uuid: "928cac1251304933a300d17de83c3229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d2ffdc2ce44df785eafca236753ac1" is_pre_election: true
I20260809 03:25:46.067826 17556 raft_consensus.cc:2454] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 928cac1251304933a300d17de83c3229 in term 0.
I20260809 03:25:46.067865 17706 raft_consensus.cc:2454] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 928cac1251304933a300d17de83c3229 in term 0.
I20260809 03:25:46.068207 17785 leader_election.cc:302] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63d2ffdc2ce44df785eafca236753ac1, 928cac1251304933a300d17de83c3229; no voters:
I20260809 03:25:46.068387 17943 raft_consensus.cc:2781] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:46.068451 17943 raft_consensus.cc:482] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:46.068502 17943 raft_consensus.cc:3037] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:46.069512 17943 raft_consensus.cc:504] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.069864 17943 leader_election.cc:288] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [CANDIDATE]: Term 1 election: Requested vote from peers 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993), 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743)
I20260809 03:25:46.070219 17556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9908ba4235fa49c9b6aae373bf079de0" candidate_uuid: "928cac1251304933a300d17de83c3229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a824b778a724ec6b332a19a3097b423"
I20260809 03:25:46.070219 17706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9908ba4235fa49c9b6aae373bf079de0" candidate_uuid: "928cac1251304933a300d17de83c3229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d2ffdc2ce44df785eafca236753ac1"
I20260809 03:25:46.070343 17556 raft_consensus.cc:3037] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:46.070343 17706 raft_consensus.cc:3037] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:46.071457 17556 raft_consensus.cc:2454] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928cac1251304933a300d17de83c3229 in term 1.
I20260809 03:25:46.071457 17706 raft_consensus.cc:2454] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928cac1251304933a300d17de83c3229 in term 1.
I20260809 03:25:46.071738 17786 leader_election.cc:302] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a824b778a724ec6b332a19a3097b423, 928cac1251304933a300d17de83c3229; no voters:
I20260809 03:25:46.071903 17943 raft_consensus.cc:2781] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:46.072016 17943 raft_consensus.cc:686] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [term 1 LEADER]: Becoming Leader. State: Replica: 928cac1251304933a300d17de83c3229, State: Running, Role: LEADER
I20260809 03:25:46.072156 17943 consensus_queue.cc:237] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.072875 17968 tablet_replica.cc:439] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: TxnStatusTablet state changed. Reason: New leader 928cac1251304933a300d17de83c3229. Latest consensus state: current_term: 1 leader_uuid: "928cac1251304933a300d17de83c3229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } }
I20260809 03:25:46.072997 17968 tablet_replica.cc:442] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:25:46.073331 17980 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 03:25:46.073509 17980 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 03:25:46.074237 17386 catalog_manager.cc:5495] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 reported cstate change: term changed from 0 to 1, leader changed from <none> to 928cac1251304933a300d17de83c3229 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "928cac1251304933a300d17de83c3229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } health_report { overall_health: HEALTHY } } }
I20260809 03:25:46.091903 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.087753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34600 (local address 127.9.46.195:39993)
0809 03:25:46.087878 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.087882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.087960 (+ 78us) server_negotiation.cc:407] Connection header received
0809 03:25:46.088168 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.088170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.088224 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.088306 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:46.088625 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.089292 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.090536 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.090749 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.090828 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.091149 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.091547 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.091639 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.091745 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.091794 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:25:46.091970 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.087642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:39993 (local address 127.0.0.1:34600)
0809 03:25:46.087885 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:46.087900 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:46.088019 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:46.088346 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:46.088349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:46.088359 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:46.088473 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.088479 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.089428 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.089431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.090431 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.090442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.090860 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.090864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.090956 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.091010 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.091644 (+ 634us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.091659 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 03:25:46.091737 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 03:25:46.091828 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260809 03:25:46.095104 17706 raft_consensus.cc:1261] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Refusing update from remote peer 928cac1251304933a300d17de83c3229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:46.095360 17556 raft_consensus.cc:1261] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Refusing update from remote peer 928cac1251304933a300d17de83c3229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:46.096001 17968 consensus_queue.cc:1035] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:46.096237 17943 consensus_queue.cc:1035] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:46.097774 17940 tablet_replica.cc:439] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: TxnStatusTablet state changed. Reason: New leader 928cac1251304933a300d17de83c3229. Latest consensus state: current_term: 1 leader_uuid: "928cac1251304933a300d17de83c3229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } }
I20260809 03:25:46.097903 17940 tablet_replica.cc:442] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:46.097884 17946 tablet_replica.cc:439] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: TxnStatusTablet state changed. Reason: New leader 928cac1251304933a300d17de83c3229. Latest consensus state: current_term: 1 leader_uuid: "928cac1251304933a300d17de83c3229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } }
I20260809 03:25:46.098001 17946 tablet_replica.cc:442] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:46.100533 17943 tablet_replica.cc:439] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "928cac1251304933a300d17de83c3229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } }
I20260809 03:25:46.100673 17943 tablet_replica.cc:442] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:25:46.100796 17943 tablet_replica.cc:439] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "928cac1251304933a300d17de83c3229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } }
I20260809 03:25:46.100864 17943 tablet_replica.cc:442] T 9908ba4235fa49c9b6aae373bf079de0 P 928cac1251304933a300d17de83c3229: This TxnStatusTablet replica's current role is: LEADER
I20260809 03:25:46.101334 17949 mvcc.cc:205] Tried to move back new op lower bound from 7316463395200180224 to 7316463395112947712. Current Snapshot: MvccSnapshot[applied={T|T < 7316463395200180224}]
I20260809 03:25:46.102116 17946 tablet_replica.cc:439] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "928cac1251304933a300d17de83c3229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } }
I20260809 03:25:46.102236 17946 tablet_replica.cc:442] T 9908ba4235fa49c9b6aae373bf079de0 P 5a824b778a724ec6b332a19a3097b423: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:46.103361 17940 tablet_replica.cc:439] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "928cac1251304933a300d17de83c3229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } } }
I20260809 03:25:46.103490 17940 tablet_replica.cc:442] T 9908ba4235fa49c9b6aae373bf079de0 P 63d2ffdc2ce44df785eafca236753ac1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 03:25:46.110358 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.106780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.9.46.195:39993)
0809 03:25:46.106891 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.106895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.106928 (+ 33us) server_negotiation.cc:407] Connection header received
0809 03:25:46.107095 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.107098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.107147 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.107228 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:46.107465 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.108251 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.109105 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.109445 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.109550 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.109833 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.109950 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.110031 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.110180 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.110229 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:25:46.121032 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.117209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:39089 (local address 127.9.46.195:39993)
0809 03:25:46.117336 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.117341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.117568 (+ 227us) server_negotiation.cc:407] Connection header received
0809 03:25:46.117705 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.117707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.117746 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.117800 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:46.118066 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.118519 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.119867 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.120105 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.120227 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.120614 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.120674 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.120733 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.120855 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.120893 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":49}
I20260809 03:25:46.121481 17988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.117174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:39993 (local address 127.9.46.195:39089)
0809 03:25:46.117480 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:46.117495 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:46.117607 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:46.117838 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:46.117841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:46.117848 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:46.117968 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.117974 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.118628 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.118631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.119639 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.119650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.120232 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.120236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.120366 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.120416 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.120759 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.120766 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 03:25:46.120837 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 03:25:46.120913 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":18,"thread_start_us":118,"threads_started":1}
I20260809 03:25:46.129119 17556 raft_consensus.cc:1261] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Refusing update from remote peer 928cac1251304933a300d17de83c3229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:46.129119 17706 raft_consensus.cc:1261] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Refusing update from remote peer 928cac1251304933a300d17de83c3229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:46.129500 17943 consensus_queue.cc:1035] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:46.129763 17985 consensus_queue.cc:1035] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:46.134030 17950 mvcc.cc:205] Tried to move back new op lower bound from 7316463395339579392 to 7316463394574954496. Current Snapshot: MvccSnapshot[applied={T|T < 7316463395339579392}]
I20260809 03:25:46.171101 17387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:46.171280 17387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:46.173316 17387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22: Sending DeleteTablet for 3 replicas of tablet acdf549c48a1415baab48278b0085bd9
I20260809 03:25:46.173846 17535 tablet_service.cc:1514] Processing DeleteTablet for tablet acdf549c48a1415baab48278b0085bd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:46 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:35654
I20260809 03:25:46.174027 17961 tablet_replica.cc:330] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: stopping tablet replica
I20260809 03:25:46.173997 17834 tablet_service.cc:1514] Processing DeleteTablet for tablet acdf549c48a1415baab48278b0085bd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:46 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:34590
I20260809 03:25:46.174151 17961 raft_consensus.cc:2229] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:46.174256 17961 raft_consensus.cc:2258] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:46.174427 17963 tablet_replica.cc:330] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: stopping tablet replica
I20260809 03:25:46.174543 17963 raft_consensus.cc:2229] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:46.174798 17963 raft_consensus.cc:2258] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:46.177932 17963 ts_tablet_manager.cc:1905] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:46.178642 17685 tablet_service.cc:1514] Processing DeleteTablet for tablet acdf549c48a1415baab48278b0085bd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:46 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:51624
I20260809 03:25:46.178834 17962 tablet_replica.cc:330] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: stopping tablet replica
I20260809 03:25:46.178980 17962 raft_consensus.cc:2229] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:46.179109 17962 raft_consensus.cc:2258] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:46.179450 17961 ts_tablet_manager.cc:1905] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:46.179878 17962 ts_tablet_manager.cc:1905] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:46.181211 17963 ts_tablet_manager.cc:1918] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:46.181281 17963 log.cc:1196] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/acdf549c48a1415baab48278b0085bd9
I20260809 03:25:46.181555 17963 ts_tablet_manager.cc:1939] T acdf549c48a1415baab48278b0085bd9 P 928cac1251304933a300d17de83c3229: Deleting consensus metadata
I20260809 03:25:46.182379 17373 catalog_manager.cc:4841] TS 928cac1251304933a300d17de83c3229 (127.9.46.195:39993): tablet acdf549c48a1415baab48278b0085bd9 (table test-table [id=c5ca89ce69924f2bbb230630e68d08f3]) successfully deleted
I20260809 03:25:46.182545 17961 ts_tablet_manager.cc:1918] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:46.182629 17961 log.cc:1196] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/acdf549c48a1415baab48278b0085bd9
I20260809 03:25:46.182717 17962 ts_tablet_manager.cc:1918] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:46.182785 17962 log.cc:1196] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/acdf549c48a1415baab48278b0085bd9
I20260809 03:25:46.182924 17961 ts_tablet_manager.cc:1939] T acdf549c48a1415baab48278b0085bd9 P 5a824b778a724ec6b332a19a3097b423: Deleting consensus metadata
I20260809 03:25:46.183075 17962 ts_tablet_manager.cc:1939] T acdf549c48a1415baab48278b0085bd9 P 63d2ffdc2ce44df785eafca236753ac1: Deleting consensus metadata
I20260809 03:25:46.183705 17374 catalog_manager.cc:4841] TS 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993): tablet acdf549c48a1415baab48278b0085bd9 (table test-table [id=c5ca89ce69924f2bbb230630e68d08f3]) successfully deleted
I20260809 03:25:46.184033 17373 catalog_manager.cc:4841] TS 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743): tablet acdf549c48a1415baab48278b0085bd9 (table test-table [id=c5ca89ce69924f2bbb230630e68d08f3]) successfully deleted
Aug 09 03:25:46 dist-test-slave-65mx krb5kdc[17339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 03:25:46 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:46 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:25:46.208151 17440 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:46.200582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.9.46.254:41909)
0809 03:25:46.200726 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.200732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.200935 (+ 203us) server_negotiation.cc:407] Connection header received
0809 03:25:46.201094 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.201096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.201144 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.201215 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:46.202159 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.202688 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.203903 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.204120 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.204201 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.207989 (+ 3788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43514: BlockingRecv error: recv got EOF from 127.0.0.1:43514 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 03:25:46 dist-test-slave-65mx krb5kdc[17339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM
I20260809 03:25:46.219219 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.209988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.9.46.254:41909)
0809 03:25:46.210114 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.210119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.210288 (+ 169us) server_negotiation.cc:407] Connection header received
0809 03:25:46.210384 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.210387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.210437 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.210530 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:46.211103 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.211905 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.212770 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.213111 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.213219 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.215727 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:46.215733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:46.215736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:46.215764 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:46.217406 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:46.217850 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:46.217854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:46.217856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:46.217948 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:46.218236 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:46.218240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:46.218241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:46.218483 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:46.218621 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.218901 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.219020 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:25:46.221984 17387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:25:46.222448 17387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:46.227484 17535 tablet_service.cc:1467] Processing CreateTablet for tablet 128cfe9c0e644edd9757365e5d63af9f (DEFAULT_TABLE table=test-table [id=4127236eb97a45d3ba79c8889f1a1943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:46.227831 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 128cfe9c0e644edd9757365e5d63af9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:46.230873 17834 tablet_service.cc:1467] Processing CreateTablet for tablet 128cfe9c0e644edd9757365e5d63af9f (DEFAULT_TABLE table=test-table [id=4127236eb97a45d3ba79c8889f1a1943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:46.231115 17934 tablet_bootstrap.cc:492] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423: Bootstrap starting.
I20260809 03:25:46.231098 17685 tablet_service.cc:1467] Processing CreateTablet for tablet 128cfe9c0e644edd9757365e5d63af9f (DEFAULT_TABLE table=test-table [id=4127236eb97a45d3ba79c8889f1a1943]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:46.231233 17834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 128cfe9c0e644edd9757365e5d63af9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:46.231365 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 128cfe9c0e644edd9757365e5d63af9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:46.232043 17934 tablet_bootstrap.cc:654] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:46.233330 17934 tablet_bootstrap.cc:492] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423: No bootstrap required, opened a new log
I20260809 03:25:46.233414 17934 ts_tablet_manager.cc:1397] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:46.233807 17934 raft_consensus.cc:348] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.233922 17934 raft_consensus.cc:374] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:46.233950 17934 raft_consensus.cc:729] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a824b778a724ec6b332a19a3097b423, State: Initialized, Role: FOLLOWER
I20260809 03:25:46.234025 17936 tablet_bootstrap.cc:492] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229: Bootstrap starting.
I20260809 03:25:46.234064 17934 consensus_queue.cc:260] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.234259 17935 tablet_bootstrap.cc:492] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1: Bootstrap starting.
I20260809 03:25:46.234325 17934 ts_tablet_manager.cc:1428] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:46.234961 17936 tablet_bootstrap.cc:654] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:46.235201 17935 tablet_bootstrap.cc:654] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:46.236722 17935 tablet_bootstrap.cc:492] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1: No bootstrap required, opened a new log
I20260809 03:25:46.236804 17935 ts_tablet_manager.cc:1397] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:25:46.237301 17935 raft_consensus.cc:348] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.237409 17935 raft_consensus.cc:374] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:46.237449 17935 raft_consensus.cc:729] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d2ffdc2ce44df785eafca236753ac1, State: Initialized, Role: FOLLOWER
I20260809 03:25:46.237571 17935 consensus_queue.cc:260] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.237843 17936 tablet_bootstrap.cc:492] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229: No bootstrap required, opened a new log
I20260809 03:25:46.237927 17936 ts_tablet_manager.cc:1397] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 03:25:46.238381 17936 raft_consensus.cc:348] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.238490 17936 raft_consensus.cc:374] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:46.238530 17936 raft_consensus.cc:729] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928cac1251304933a300d17de83c3229, State: Initialized, Role: FOLLOWER
I20260809 03:25:46.238777 17935 ts_tablet_manager.cc:1428] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 03:25:46.238777 17936 consensus_queue.cc:260] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.239161 17936 ts_tablet_manager.cc:1428] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:46.364138 17946 raft_consensus.cc:482] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:46.364321 17946 raft_consensus.cc:504] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.364845 17946 leader_election.cc:288] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743), 928cac1251304933a300d17de83c3229 (127.9.46.195:39993)
I20260809 03:25:46.365331 17706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128cfe9c0e644edd9757365e5d63af9f" candidate_uuid: "5a824b778a724ec6b332a19a3097b423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d2ffdc2ce44df785eafca236753ac1" is_pre_election: true
I20260809 03:25:46.365409 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128cfe9c0e644edd9757365e5d63af9f" candidate_uuid: "5a824b778a724ec6b332a19a3097b423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928cac1251304933a300d17de83c3229" is_pre_election: true
I20260809 03:25:46.365499 17706 raft_consensus.cc:2454] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a824b778a724ec6b332a19a3097b423 in term 0.
I20260809 03:25:46.365567 17856 raft_consensus.cc:2454] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a824b778a724ec6b332a19a3097b423 in term 0.
I20260809 03:25:46.365839 17485 leader_election.cc:302] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a824b778a724ec6b332a19a3097b423, 928cac1251304933a300d17de83c3229; no voters:
I20260809 03:25:46.366056 17946 raft_consensus.cc:2781] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:46.366117 17946 raft_consensus.cc:482] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:46.366164 17946 raft_consensus.cc:3037] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:46.367154 17946 raft_consensus.cc:504] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.367509 17946 leader_election.cc:288] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [CANDIDATE]: Term 1 election: Requested vote from peers 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743), 928cac1251304933a300d17de83c3229 (127.9.46.195:39993)
I20260809 03:25:46.367888 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128cfe9c0e644edd9757365e5d63af9f" candidate_uuid: "5a824b778a724ec6b332a19a3097b423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928cac1251304933a300d17de83c3229"
I20260809 03:25:46.367880 17706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "128cfe9c0e644edd9757365e5d63af9f" candidate_uuid: "5a824b778a724ec6b332a19a3097b423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d2ffdc2ce44df785eafca236753ac1"
I20260809 03:25:46.368000 17706 raft_consensus.cc:3037] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:46.368000 17856 raft_consensus.cc:3037] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:46.369171 17706 raft_consensus.cc:2454] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a824b778a724ec6b332a19a3097b423 in term 1.
I20260809 03:25:46.369171 17856 raft_consensus.cc:2454] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a824b778a724ec6b332a19a3097b423 in term 1.
I20260809 03:25:46.369482 17485 leader_election.cc:302] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a824b778a724ec6b332a19a3097b423, 63d2ffdc2ce44df785eafca236753ac1; no voters:
I20260809 03:25:46.369715 17946 raft_consensus.cc:2781] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:46.369935 17946 raft_consensus.cc:686] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 1 LEADER]: Becoming Leader. State: Replica: 5a824b778a724ec6b332a19a3097b423, State: Running, Role: LEADER
I20260809 03:25:46.370096 17946 consensus_queue.cc:237] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.371861 17389 catalog_manager.cc:5495] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a824b778a724ec6b332a19a3097b423 (127.9.46.193). New cstate: current_term: 1 leader_uuid: "5a824b778a724ec6b332a19a3097b423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } health_report { overall_health: UNKNOWN } } }
I20260809 03:25:46.393610 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.390078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35680 (local address 127.9.46.193:35993)
0809 03:25:46.390224 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.390229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.390246 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:25:46.390342 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.390345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.390404 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.390495 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:46.390740 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.391577 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.392415 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.392836 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.392942 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.393175 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.393256 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.393324 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.393405 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.393462 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:25:46.395787 17706 raft_consensus.cc:1261] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Refusing update from remote peer 5a824b778a724ec6b332a19a3097b423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:46.396010 17856 raft_consensus.cc:1261] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Refusing update from remote peer 5a824b778a724ec6b332a19a3097b423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:46.396240 18009 consensus_queue.cc:1035] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:46.396437 17946 consensus_queue.cc:1035] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:46.405608 17387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:46.405759 17387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:46.407848 17387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22: Sending DeleteTablet for 3 replicas of tablet 128cfe9c0e644edd9757365e5d63af9f
I20260809 03:25:46.408607 17834 tablet_service.cc:1514] Processing DeleteTablet for tablet 128cfe9c0e644edd9757365e5d63af9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:46 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:34590
I20260809 03:25:46.408617 17685 tablet_service.cc:1514] Processing DeleteTablet for tablet 128cfe9c0e644edd9757365e5d63af9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:46 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:51624
I20260809 03:25:46.408797 17963 tablet_replica.cc:330] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229: stopping tablet replica
I20260809 03:25:46.408835 17962 tablet_replica.cc:330] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1: stopping tablet replica
I20260809 03:25:46.408907 17963 raft_consensus.cc:2229] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:46.408963 17962 raft_consensus.cc:2229] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:46.409034 17963 raft_consensus.cc:2258] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:46.409088 17962 raft_consensus.cc:2258] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:46.409411 17535 tablet_service.cc:1514] Processing DeleteTablet for tablet 128cfe9c0e644edd9757365e5d63af9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:46 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:35654
I20260809 03:25:46.409590 17961 tablet_replica.cc:330] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423: stopping tablet replica
I20260809 03:25:46.409715 17963 ts_tablet_manager.cc:1905] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:46.409737 17961 raft_consensus.cc:2229] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:46.409873 17962 ts_tablet_manager.cc:1905] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:46.409931 17961 raft_consensus.cc:2258] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:46.410758 17961 ts_tablet_manager.cc:1905] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:46.412245 17962 ts_tablet_manager.cc:1918] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:46.412266 17963 ts_tablet_manager.cc:1918] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:46.412168 17387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 03:25:46.412330 17963 log.cc:1196] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/128cfe9c0e644edd9757365e5d63af9f
I20260809 03:25:46.412328 17962 log.cc:1196] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/128cfe9c0e644edd9757365e5d63af9f
I20260809 03:25:46.412639 17962 ts_tablet_manager.cc:1939] T 128cfe9c0e644edd9757365e5d63af9f P 63d2ffdc2ce44df785eafca236753ac1: Deleting consensus metadata
I20260809 03:25:46.412648 17963 ts_tablet_manager.cc:1939] T 128cfe9c0e644edd9757365e5d63af9f P 928cac1251304933a300d17de83c3229: Deleting consensus metadata
W20260809 03:25:46.412643 17387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:25:46.413504 17961 ts_tablet_manager.cc:1918] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:25:46.413515 17373 catalog_manager.cc:4841] TS 928cac1251304933a300d17de83c3229 (127.9.46.195:39993): tablet 128cfe9c0e644edd9757365e5d63af9f (table test-table [id=4127236eb97a45d3ba79c8889f1a1943]) successfully deleted
I20260809 03:25:46.413594 17961 log.cc:1196] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/128cfe9c0e644edd9757365e5d63af9f
I20260809 03:25:46.413659 17373 catalog_manager.cc:4841] TS 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743): tablet 128cfe9c0e644edd9757365e5d63af9f (table test-table [id=4127236eb97a45d3ba79c8889f1a1943]) successfully deleted
I20260809 03:25:46.413882 17961 ts_tablet_manager.cc:1939] T 128cfe9c0e644edd9757365e5d63af9f P 5a824b778a724ec6b332a19a3097b423: Deleting consensus metadata
I20260809 03:25:46.414564 17374 catalog_manager.cc:4841] TS 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993): tablet 128cfe9c0e644edd9757365e5d63af9f (table test-table [id=4127236eb97a45d3ba79c8889f1a1943]) successfully deleted
I20260809 03:25:46.417486 17535 tablet_service.cc:1467] Processing CreateTablet for tablet 3544319eebb5419484a36968aab0089a (DEFAULT_TABLE table=test-table [id=657448e4eee34c538f534fc1799f579d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:46.417533 17685 tablet_service.cc:1467] Processing CreateTablet for tablet 3544319eebb5419484a36968aab0089a (DEFAULT_TABLE table=test-table [id=657448e4eee34c538f534fc1799f579d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:46.417601 17834 tablet_service.cc:1467] Processing CreateTablet for tablet 3544319eebb5419484a36968aab0089a (DEFAULT_TABLE table=test-table [id=657448e4eee34c538f534fc1799f579d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:25:46.417769 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3544319eebb5419484a36968aab0089a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:46.417796 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3544319eebb5419484a36968aab0089a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:46.417830 17834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3544319eebb5419484a36968aab0089a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:46.419719 17934 tablet_bootstrap.cc:492] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423: Bootstrap starting.
I20260809 03:25:46.420027 17935 tablet_bootstrap.cc:492] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1: Bootstrap starting.
I20260809 03:25:46.420606 17934 tablet_bootstrap.cc:654] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:46.420996 17935 tablet_bootstrap.cc:654] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:46.421200 17936 tablet_bootstrap.cc:492] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229: Bootstrap starting.
I20260809 03:25:46.421986 17936 tablet_bootstrap.cc:654] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:46.422267 17935 tablet_bootstrap.cc:492] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1: No bootstrap required, opened a new log
I20260809 03:25:46.422348 17935 ts_tablet_manager.cc:1397] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:46.422811 17935 raft_consensus.cc:348] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.422941 17935 raft_consensus.cc:374] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:46.422981 17935 raft_consensus.cc:729] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d2ffdc2ce44df785eafca236753ac1, State: Initialized, Role: FOLLOWER
I20260809 03:25:46.423130 17935 consensus_queue.cc:260] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.423353 17936 tablet_bootstrap.cc:492] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229: No bootstrap required, opened a new log
I20260809 03:25:46.423429 17935 ts_tablet_manager.cc:1428] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:25:46.423439 17936 ts_tablet_manager.cc:1397] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 03:25:46.423799 17936 raft_consensus.cc:348] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.423920 17936 raft_consensus.cc:374] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:46.423974 17936 raft_consensus.cc:729] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928cac1251304933a300d17de83c3229, State: Initialized, Role: FOLLOWER
I20260809 03:25:46.424100 17936 consensus_queue.cc:260] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.424357 17936 ts_tablet_manager.cc:1428] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:25:46.425163 17934 tablet_bootstrap.cc:492] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423: No bootstrap required, opened a new log
I20260809 03:25:46.425258 17934 ts_tablet_manager.cc:1397] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260809 03:25:46.425704 17934 raft_consensus.cc:348] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.425841 17934 raft_consensus.cc:374] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:46.425879 17934 raft_consensus.cc:729] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a824b778a724ec6b332a19a3097b423, State: Initialized, Role: FOLLOWER
I20260809 03:25:46.426002 17934 consensus_queue.cc:260] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.426273 17934 ts_tablet_manager.cc:1428] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 03:25:46.682087 17940 raft_consensus.cc:482] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:25:46.682245 17940 raft_consensus.cc:504] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.683540 17940 leader_election.cc:288] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993), 928cac1251304933a300d17de83c3229 (127.9.46.195:39993)
I20260809 03:25:46.689272 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.683956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:41655 (local address 127.9.46.193:35993)
0809 03:25:46.684147 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.684154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.684468 (+ 314us) server_negotiation.cc:407] Connection header received
0809 03:25:46.684618 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.684621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.684671 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.684781 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:46.685157 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.685963 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.687829 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.688476 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.688705 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.688754 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.689037 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.689133 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 03:25:46.689273 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.684072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:51211 (local address 127.9.46.195:39993)
0809 03:25:46.684336 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.684343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.684419 (+ 76us) server_negotiation.cc:407] Connection header received
0809 03:25:46.684612 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.684616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.684671 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.684779 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:25:46.685166 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.685963 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.687829 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.688483 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.688681 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.688744 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.689036 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.689139 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
I20260809 03:25:46.689299 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.683849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:39993 (local address 127.9.46.194:51211)
0809 03:25:46.684294 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:46.684312 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:46.684464 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:46.684812 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:46.684818 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:46.684843 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:46.685008 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.685015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.686130 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.686133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.687636 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.687650 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.688680 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.688684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.688799 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.688884 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 03:25:46.688972 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 03:25:46.689103 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":186,"thread_start_us":99,"threads_started":1}
I20260809 03:25:46.689309 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.683849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:35993 (local address 127.9.46.194:41655)
0809 03:25:46.684400 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:46.684413 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:46.684490 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:46.684843 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:46.684846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:46.684858 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:25:46.684994 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.685001 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.686130 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.686133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.687642 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.687650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.688719 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.688722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.688821 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.688889 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 03:25:46.688972 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:25:46.689103 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":325,"thread_start_us":55,"threads_started":1}
I20260809 03:25:46.689828 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3544319eebb5419484a36968aab0089a" candidate_uuid: "63d2ffdc2ce44df785eafca236753ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928cac1251304933a300d17de83c3229" is_pre_election: true
I20260809 03:25:46.689991 17856 raft_consensus.cc:2454] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63d2ffdc2ce44df785eafca236753ac1 in term 0.
I20260809 03:25:46.690469 17555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3544319eebb5419484a36968aab0089a" candidate_uuid: "63d2ffdc2ce44df785eafca236753ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a824b778a724ec6b332a19a3097b423" is_pre_election: true
I20260809 03:25:46.690469 17632 leader_election.cc:302] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63d2ffdc2ce44df785eafca236753ac1, 928cac1251304933a300d17de83c3229; no voters:
I20260809 03:25:46.690630 17555 raft_consensus.cc:2454] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63d2ffdc2ce44df785eafca236753ac1 in term 0.
I20260809 03:25:46.690778 17940 raft_consensus.cc:2781] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:25:46.690841 17940 raft_consensus.cc:482] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:25:46.690891 17940 raft_consensus.cc:3037] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:46.691985 17940 raft_consensus.cc:504] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.692411 17940 leader_election.cc:288] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [CANDIDATE]: Term 1 election: Requested vote from peers 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993), 928cac1251304933a300d17de83c3229 (127.9.46.195:39993)
I20260809 03:25:46.692809 17555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3544319eebb5419484a36968aab0089a" candidate_uuid: "63d2ffdc2ce44df785eafca236753ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a824b778a724ec6b332a19a3097b423"
I20260809 03:25:46.692946 17555 raft_consensus.cc:3037] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:46.692957 17856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3544319eebb5419484a36968aab0089a" candidate_uuid: "63d2ffdc2ce44df785eafca236753ac1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928cac1251304933a300d17de83c3229"
I20260809 03:25:46.693114 17856 raft_consensus.cc:3037] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:46.693979 17555 raft_consensus.cc:2454] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63d2ffdc2ce44df785eafca236753ac1 in term 1.
I20260809 03:25:46.694080 17856 raft_consensus.cc:2454] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63d2ffdc2ce44df785eafca236753ac1 in term 1.
I20260809 03:25:46.694324 17633 leader_election.cc:302] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a824b778a724ec6b332a19a3097b423, 63d2ffdc2ce44df785eafca236753ac1; no voters:
I20260809 03:25:46.694578 17940 raft_consensus.cc:2781] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:46.694860 17940 raft_consensus.cc:686] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 1 LEADER]: Becoming Leader. State: Replica: 63d2ffdc2ce44df785eafca236753ac1, State: Running, Role: LEADER
I20260809 03:25:46.695144 17940 consensus_queue.cc:237] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } }
I20260809 03:25:46.698010 17386 catalog_manager.cc:5495] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194). New cstate: current_term: 1 leader_uuid: "63d2ffdc2ce44df785eafca236753ac1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63d2ffdc2ce44df785eafca236753ac1" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 } health_report { overall_health: UNKNOWN } } }
I20260809 03:25:46.725723 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.721898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.9.46.194:39743)
0809 03:25:46.722220 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.722228 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.722238 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:25:46.722307 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.722309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.722359 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.722492 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:46.722763 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.723308 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.724152 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.724400 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.724510 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.724829 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.725295 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.725427 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.725495 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.725553 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":117,"threads_started":1}
I20260809 03:25:46.736248 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.733159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:38153 (local address 127.9.46.195:39993)
0809 03:25:46.733334 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.733338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.733430 (+ 92us) server_negotiation.cc:407] Connection header received
0809 03:25:46.733547 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.733550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.733610 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.733677 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:46.733926 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.734426 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.735297 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.735511 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.735606 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.735859 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.735928 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.735983 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.736080 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.736131 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 03:25:46.736282 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.733013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:39993 (local address 127.9.46.194:38153)
0809 03:25:46.733349 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:46.733363 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:46.733458 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:46.733706 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:46.733708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:46.733714 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:46.733833 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.733838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.734533 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.734535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.735182 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.735192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.735617 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.735620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.735691 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.735739 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.736005 (+ 266us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.736010 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:25:46.736061 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 03:25:46.736151 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":117,"threads_started":1}
I20260809 03:25:46.742910 17856 raft_consensus.cc:1261] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Refusing update from remote peer 63d2ffdc2ce44df785eafca236753ac1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:46.742908 17555 raft_consensus.cc:1261] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Refusing update from remote peer 63d2ffdc2ce44df785eafca236753ac1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:25:46.743498 17940 consensus_queue.cc:1035] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a824b778a724ec6b332a19a3097b423" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 35993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:46.744131 18020 consensus_queue.cc:1035] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928cac1251304933a300d17de83c3229" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 39993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:25:46.762986 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.759412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:51781 (local address 127.9.46.194:39743)
0809 03:25:46.759614 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:46.759618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:46.759748 (+ 130us) server_negotiation.cc:407] Connection header received
0809 03:25:46.759977 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:46.759979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:46.760024 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:46.760093 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:25:46.760285 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.760864 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.761819 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.762094 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.762200 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.762464 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.762551 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.762665 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:46.762770 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 03:25:46.762840 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260809 03:25:46.763020 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.759345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:39743 (local address 127.9.46.195:51781)
0809 03:25:46.759626 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:46.759643 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:46.759765 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:46.760119 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:46.760121 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:46.760127 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 03:25:46.760215 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.760219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.760960 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.760962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.761680 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:46.761689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:46.762213 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:46.762217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:46.762306 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:46.762337 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 03:25:46.762682 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 03:25:46.762690 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:25:46.762761 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 03:25:46.762855 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":114,"threads_started":1}
I20260809 03:25:46.785733 17389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:46.785885 17389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:25:46.788180 17389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c431a8aa3764054a31da9409773da22: Sending DeleteTablet for 3 replicas of tablet 3544319eebb5419484a36968aab0089a
I20260809 03:25:46.788973 17535 tablet_service.cc:1514] Processing DeleteTablet for tablet 3544319eebb5419484a36968aab0089a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:46 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:35654
I20260809 03:25:46.788964 17834 tablet_service.cc:1514] Processing DeleteTablet for tablet 3544319eebb5419484a36968aab0089a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:46 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:34590
I20260809 03:25:46.789196 17961 tablet_replica.cc:330] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423: stopping tablet replica
I20260809 03:25:46.789350 17961 raft_consensus.cc:2229] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:46.789491 17961 raft_consensus.cc:2258] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:46.789497 17685 tablet_service.cc:1514] Processing DeleteTablet for tablet 3544319eebb5419484a36968aab0089a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:25:46 UTC) from {username='oryx', principal='oryx/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:51624
I20260809 03:25:46.789685 17963 tablet_replica.cc:330] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229: stopping tablet replica
I20260809 03:25:46.789732 17962 tablet_replica.cc:330] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1: stopping tablet replica
I20260809 03:25:46.789819 17963 raft_consensus.cc:2229] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:25:46.789847 17962 raft_consensus.cc:2229] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:25:46.789974 17963 raft_consensus.cc:2258] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:46.790177 17962 raft_consensus.cc:2258] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:25:46.790629 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 17466
I20260809 03:25:46.790920 17961 ts_tablet_manager.cc:1905] T 3544319eebb5419484a36968aab0089a P 5a824b778a724ec6b332a19a3097b423: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:46.791013 17962 ts_tablet_manager.cc:1905] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:46.793758 17962 ts_tablet_manager.cc:1918] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:46.793838 17962 log.cc:1196] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/3544319eebb5419484a36968aab0089a
I20260809 03:25:46.794119 17962 ts_tablet_manager.cc:1939] T 3544319eebb5419484a36968aab0089a P 63d2ffdc2ce44df785eafca236753ac1: Deleting consensus metadata
I20260809 03:25:46.794832 17373 catalog_manager.cc:4841] TS 63d2ffdc2ce44df785eafca236753ac1 (127.9.46.194:39743): tablet 3544319eebb5419484a36968aab0089a (table test-table [id=657448e4eee34c538f534fc1799f579d]) successfully deleted
I20260809 03:25:46.796528 17963 ts_tablet_manager.cc:1905] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:25:46.798906 17963 ts_tablet_manager.cc:1918] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 03:25:46.799021 17963 log.cc:1196] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/3544319eebb5419484a36968aab0089a
I20260809 03:25:46.799268 17963 ts_tablet_manager.cc:1939] T 3544319eebb5419484a36968aab0089a P 928cac1251304933a300d17de83c3229: Deleting consensus metadata
I20260809 03:25:46.800266 17373 catalog_manager.cc:4841] TS 928cac1251304933a300d17de83c3229 (127.9.46.195:39993): tablet 3544319eebb5419484a36968aab0089a (table test-table [id=657448e4eee34c538f534fc1799f579d]) successfully deleted
W20260809 03:25:46.804795 17374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:35993 (error 108)
I20260809 03:25:46.806030 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:46.805649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:35993 (local address 127.0.0.1:35692)
0809 03:25:46.805872 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:46.805939 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:35993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":60,"threads_started":1}
I20260809 03:25:46.806182 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 17613
W20260809 03:25:46.808972 17374 catalog_manager.cc:4584] TS 5a824b778a724ec6b332a19a3097b423 (127.9.46.193:35993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3544319eebb5419484a36968aab0089a: Network error: Client connection negotiation failed: client connection to 127.9.46.193:35993: connect: Connection refused (error 111)
I20260809 03:25:46.815492 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 17768
I20260809 03:25:46.824124 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 17353
2026-08-09T03:25:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3559 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:46 dist-test-slave-65mx krb5kdc[18039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:46 dist-test-slave-65mx krb5kdc[18039](info): set up 2 sockets
Aug 09 03:25:46 dist-test-slave-65mx krb5kdc[18039](info): commencing operation
krb5kdc: starting...
W20260809 03:25:47.923983 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:48 dist-test-slave-65mx krb5kdc[18039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:48.060264 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41441
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:41441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:48.182111 18052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:48.182432 18052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:48.182515 18052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:48.186631 18052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:48.186733 18052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:48.186776 18052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:48.186836 18052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:48.186893 18052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:48.192487 18052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:41441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:48.193643 18052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:48.194702 18052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:48.203377 18060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:48.203377 18057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:48.203382 18058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:48.203693 18052 server_base.cc:1034] running on GCE node
I20260809 03:25:48.204321 18052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:48.205574 18052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:48.206781 18052 hybrid_clock.cc:648] HybridClock initialized: now 1786245948206762 us; error 45 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T03:25:48Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1391 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:25:48 dist-test-slave-65mx krb5kdc[18066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:25:48 dist-test-slave-65mx krb5kdc[18066](info): set up 2 sockets
Aug 09 03:25:48 dist-test-slave-65mx krb5kdc[18066](info): commencing operation
krb5kdc: starting...
W20260809 03:25:49.303094 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:25:49 dist-test-slave-65mx krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:25:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:25:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:25:49.440187 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:37495
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:37495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:49.556263 18080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:49.556705 18080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:49.556759 18080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:49.560915 18080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:49.560994 18080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:49.561014 18080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:49.561035 18080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:49.561051 18080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:49.566692 18080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:37495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:37495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:49.567916 18080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:49.568991 18080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:49.577898 18085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:49.578015 18086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:49.578109 18080 server_base.cc:1034] running on GCE node
W20260809 03:25:49.577899 18088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:49.578835 18080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:49.579939 18080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:49.581154 18080 hybrid_clock.cc:648] HybridClock initialized: now 1786245949581136 us; error 34 us; skew 500 ppm
Aug 09 03:25:49 dist-test-slave-65mx krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245949, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:49.584197 18080 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:49.585266 18080 webserver.cc:460] Webserver started at http://127.9.46.254:42267/ using document root <none> and password file <none>
I20260809 03:25:49.585852 18080 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:49.585916 18080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:49.586167 18080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:49.588380 18080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "297d565cdb9046349acb06413613902f"
format_stamp: "Formatted at 2026-08-09 03:25:49 on dist-test-slave-65mx"
server_key: "d72873cbc7b5084074a02d199bddd67f"
server_key_iv: "a3011780122c38736216fec62c8785fb"
server_key_version: "encryptionkey@0"
I20260809 03:25:49.589193 18080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "297d565cdb9046349acb06413613902f"
format_stamp: "Formatted at 2026-08-09 03:25:49 on dist-test-slave-65mx"
server_key: "d72873cbc7b5084074a02d199bddd67f"
server_key_iv: "a3011780122c38736216fec62c8785fb"
server_key_version: "encryptionkey@0"
I20260809 03:25:49.593511 18080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:25:49.596203 18095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:49.597797 18080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:49.597980 18080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "297d565cdb9046349acb06413613902f"
format_stamp: "Formatted at 2026-08-09 03:25:49 on dist-test-slave-65mx"
server_key: "d72873cbc7b5084074a02d199bddd67f"
server_key_iv: "a3011780122c38736216fec62c8785fb"
server_key_version: "encryptionkey@0"
I20260809 03:25:49.598122 18080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:49.616370 18080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:49.619653 18080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:49.619882 18080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:49.627619 18080 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:37495
I20260809 03:25:49.627668 18152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:37495 every 8 connection(s)
I20260809 03:25:49.628750 18080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:49.631609 18153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:25:49.637132 18153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f: Bootstrap starting.
I20260809 03:25:49.638089 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 18080
I20260809 03:25:49.638196 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:25:49.638540 9403 external_mini_cluster.cc:1428] Setting key fd0259e1ed9f226a5e8a0733b1f7fc55
I20260809 03:25:49.640045 18153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f: Neither blocks nor log segments found. Creating new log.
I20260809 03:25:49.640934 18153 log.cc:824] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:49.642894 18153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f: No bootstrap required, opened a new log
I20260809 03:25:49.646672 18153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } }
I20260809 03:25:49.646970 18153 raft_consensus.cc:374] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:25:49.647044 18153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297d565cdb9046349acb06413613902f, State: Initialized, Role: FOLLOWER
Aug 09 03:25:49 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:49.647580 18153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } }
I20260809 03:25:49.647738 18153 raft_consensus.cc:388] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:49.647804 18153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:49.647912 18153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:25:49.648960 18153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } }
I20260809 03:25:49.649312 18153 leader_election.cc:302] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 297d565cdb9046349acb06413613902f; no voters:
I20260809 03:25:49.649690 18153 leader_election.cc:288] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:25:49.649777 18159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:25:49.649995 18159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 1 LEADER]: Becoming Leader. State: Replica: 297d565cdb9046349acb06413613902f, State: Running, Role: LEADER
I20260809 03:25:49.650362 18159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } }
I20260809 03:25:49.651054 18153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:49.652124 18160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "297d565cdb9046349acb06413613902f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } } }
I20260809 03:25:49.652258 18160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:49.652690 18161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 297d565cdb9046349acb06413613902f. Latest consensus state: current_term: 1 leader_uuid: "297d565cdb9046349acb06413613902f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } } }
I20260809 03:25:49.652885 18161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:49.653386 18168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:49.654672 18156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:49.639786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37622 (local address 127.9.46.254:37495)
0809 03:25:49.640275 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:49.640292 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:49.640327 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:25:49.641080 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:49.641107 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:49.641454 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:49.641790 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:49.642647 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:49.643677 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:49.644589 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:49.645086 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:49.645266 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:49.648277 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:49.648283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:49.648296 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:49.648324 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:49.650498 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:49.650960 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:49.650964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:49.650968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:49.651078 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:49.651476 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:49.651480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:49.651482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:49.651997 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:49.652968 (+ 971us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:49.653142 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 03:25:49.653716 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":175,"threads_started":1}
I20260809 03:25:49.657542 18168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:49.663664 18168 catalog_manager.cc:1347] Generated new cluster ID: 36640619ed3f4f03b2e72fe516878660
I20260809 03:25:49.663750 18168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:49.706985 18168 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:25:49.708359 18168 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:49.726907 18168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f: Generated new TSK 0
I20260809 03:25:49.727777 18168 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:25:49.760376 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:37495
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:49.900524 18181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:49.900916 18181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:49.901002 18181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:49.905900 18181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:49.906023 18181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:49.906131 18181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:49.912377 18181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:37495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:49.913689 18181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:49.914825 18181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:49.925382 18188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:49.925382 18187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:49.925385 18190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:49.925788 18181 server_base.cc:1034] running on GCE node
I20260809 03:25:49.926204 18181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:49.926868 18181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:49.928107 18181 hybrid_clock.cc:648] HybridClock initialized: now 1786245949928092 us; error 51 us; skew 500 ppm
Aug 09 03:25:49 dist-test-slave-65mx krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245949, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:49.931027 18181 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:25:49.932014 18181 webserver.cc:460] Webserver started at http://127.9.46.193:41381/ using document root <none> and password file <none>
I20260809 03:25:49.932668 18181 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:49.932760 18181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:49.932981 18181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:49.935163 18181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "23b7fd781cb24e2bbe9c8d474b5e4b73"
format_stamp: "Formatted at 2026-08-09 03:25:49 on dist-test-slave-65mx"
server_key: "23231890e7879f1a7008d2a03134b66f"
server_key_iv: "d1eb6367a88a71be737b7f193fbb5e1e"
server_key_version: "encryptionkey@0"
I20260809 03:25:49.935812 18181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "23b7fd781cb24e2bbe9c8d474b5e4b73"
format_stamp: "Formatted at 2026-08-09 03:25:49 on dist-test-slave-65mx"
server_key: "23231890e7879f1a7008d2a03134b66f"
server_key_iv: "d1eb6367a88a71be737b7f193fbb5e1e"
server_key_version: "encryptionkey@0"
I20260809 03:25:49.940215 18181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:25:49.943104 18198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:49.944761 18181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 03:25:49.944924 18181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "23b7fd781cb24e2bbe9c8d474b5e4b73"
format_stamp: "Formatted at 2026-08-09 03:25:49 on dist-test-slave-65mx"
server_key: "23231890e7879f1a7008d2a03134b66f"
server_key_iv: "d1eb6367a88a71be737b7f193fbb5e1e"
server_key_version: "encryptionkey@0"
I20260809 03:25:49.945065 18181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:49.967164 18181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:49.970947 18181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:49.971186 18181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:49.971841 18181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:49.972803 18181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:49.972875 18181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:49.972949 18181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:49.973001 18181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:49.983044 18181 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:38453
I20260809 03:25:49.983072 18321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:38453 every 8 connection(s)
I20260809 03:25:49.984045 18181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:49.989856 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 18181
I20260809 03:25:49.990003 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:25:49.990326 9403 external_mini_cluster.cc:1428] Setting key 090932bacdadb5305a22f88a1b1e9c45
Aug 09 03:25:49 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245949, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:50.000679 18156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:49.985958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:41269 (local address 127.9.46.254:37495)
0809 03:25:49.986217 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:49.986221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:49.987014 (+ 793us) server_negotiation.cc:407] Connection header received
0809 03:25:49.988305 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:49.988308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:49.988358 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:49.988438 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:49.989870 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:49.990578 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:49.992036 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:49.992298 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:49.992381 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:49.995794 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:49.995801 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:49.995807 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:49.995842 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:49.997665 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:49.998289 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:49.998296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:49.998300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:49.998405 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:49.998955 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:49.998960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:49.998962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:49.999215 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:49.999299 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:50.000204 (+ 905us) server_negotiation.cc:299] Negotiation successful
0809 03:25:50.000419 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260809 03:25:50.001765 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:49.986326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:41269)
0809 03:25:49.986831 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:49.986880 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:49.988088 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:49.988706 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:49.988713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:49.989091 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:49.989691 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:49.989714 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:49.990719 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:49.990723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:49.991907 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:49.991923 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:49.992472 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:49.992479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:49.992780 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:49.993559 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:49.993586 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:49.995600 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:49.997819 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:49.997825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:49.997838 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:49.998154 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:49.998567 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:49.998570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:49.998572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:49.998815 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:49.999402 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:49.999410 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:49.999820 (+ 410us) client_negotiation.cc:769] Sending connection context
0809 03:25:50.000130 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 03:25:50.000510 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:25:50.003427 18322 heartbeater.cc:347] Connected to a master server at 127.9.46.254:37495
I20260809 03:25:50.003777 18322 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:50.004395 18322 heartbeater.cc:511] Master 127.9.46.254:37495 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
I20260809 03:25:50.006158 18115 ts_manager.cc:194] Registered new tserver with Master: 23b7fd781cb24e2bbe9c8d474b5e4b73 (127.9.46.193:38453)
I20260809 03:25:50.007323 18115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:41269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:25:50.023998 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:37495
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:50.139919 18327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:50.140202 18327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:50.140247 18327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:50.144327 18327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:50.144397 18327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:50.144455 18327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:50.149950 18327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:37495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:50.151122 18327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:50.152230 18327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:50.162931 18333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:50.162954 18335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:50.162941 18332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:50.163216 18327 server_base.cc:1034] running on GCE node
I20260809 03:25:50.163792 18327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:50.164443 18327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:50.165660 18327 hybrid_clock.cc:648] HybridClock initialized: now 1786245950165644 us; error 30 us; skew 500 ppm
Aug 09 03:25:50 dist-test-slave-65mx krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:50.168828 18327 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:25:50.169773 18327 webserver.cc:460] Webserver started at http://127.9.46.194:45157/ using document root <none> and password file <none>
I20260809 03:25:50.170356 18327 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:50.170452 18327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:50.170702 18327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:50.172914 18327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "4a3f1187e8ad454eb9ce0bfd7add57c7"
format_stamp: "Formatted at 2026-08-09 03:25:50 on dist-test-slave-65mx"
server_key: "cbe9ec38b8ccac540ff602ffe546df4c"
server_key_iv: "ca07fecd611b3354138f5a3e7708fcc8"
server_key_version: "encryptionkey@0"
I20260809 03:25:50.173494 18327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "4a3f1187e8ad454eb9ce0bfd7add57c7"
format_stamp: "Formatted at 2026-08-09 03:25:50 on dist-test-slave-65mx"
server_key: "cbe9ec38b8ccac540ff602ffe546df4c"
server_key_iv: "ca07fecd611b3354138f5a3e7708fcc8"
server_key_version: "encryptionkey@0"
I20260809 03:25:50.177292 18327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:25:50.179726 18342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:50.181083 18327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 03:25:50.181241 18327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "4a3f1187e8ad454eb9ce0bfd7add57c7"
format_stamp: "Formatted at 2026-08-09 03:25:50 on dist-test-slave-65mx"
server_key: "cbe9ec38b8ccac540ff602ffe546df4c"
server_key_iv: "ca07fecd611b3354138f5a3e7708fcc8"
server_key_version: "encryptionkey@0"
I20260809 03:25:50.181361 18327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:50.198737 18327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:50.202450 18327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:50.202764 18327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:50.203586 18327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:50.204497 18327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:50.204603 18327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:50.204679 18327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:50.204713 18327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:50.214180 18327 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:43899
I20260809 03:25:50.214213 18463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:43899 every 8 connection(s)
I20260809 03:25:50.215112 18327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:50.222131 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 18327
I20260809 03:25:50.222247 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:25:50.222566 9403 external_mini_cluster.cc:1428] Setting key e1c3c61292e6867e25dc28d5cf6cf566
Aug 09 03:25:50 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:50.229905 18156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:50.217089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:46325 (local address 127.9.46.254:37495)
0809 03:25:50.217255 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:50.217265 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:50.218058 (+ 793us) server_negotiation.cc:407] Connection header received
0809 03:25:50.219374 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:50.219378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:50.219438 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:50.219560 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:50.220785 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:50.221316 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:50.222782 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:50.223075 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:50.223195 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:50.226074 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:50.226078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:50.226081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:50.226110 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:50.227689 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:50.228224 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:50.228228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:50.228230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:50.228316 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:50.228755 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:50.228759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:50.228761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:50.228953 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:50.229067 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:50.229595 (+ 528us) server_negotiation.cc:299] Negotiation successful
0809 03:25:50.229732 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 03:25:50.230737 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:50.217402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:46325)
0809 03:25:50.217853 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:50.217896 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:50.219158 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:50.219733 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:50.219740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:50.220121 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:50.220633 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:50.220647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:50.221452 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:50.221456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:50.222643 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:50.222657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:50.223359 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:50.223362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:50.223508 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:50.224165 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:50.224187 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:50.225870 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:50.227824 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:50.227835 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:50.227848 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:50.228114 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:50.228413 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:50.228417 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:50.228419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:50.228650 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:50.229076 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:50.229085 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:50.229336 (+ 251us) client_negotiation.cc:769] Sending connection context
0809 03:25:50.229527 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 03:25:50.229865 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":113,"threads_started":1}
I20260809 03:25:50.231962 18464 heartbeater.cc:347] Connected to a master server at 127.9.46.254:37495
I20260809 03:25:50.232249 18464 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:50.232913 18464 heartbeater.cc:511] Master 127.9.46.254:37495 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:25:50.234159 18115 ts_manager.cc:194] Registered new tserver with Master: 4a3f1187e8ad454eb9ce0bfd7add57c7 (127.9.46.194:43899)
I20260809 03:25:50.234815 18115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:46325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:25:50.255414 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:37495
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:50.372722 18469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:50.373016 18469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:50.373096 18469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:50.377310 18469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:50.377377 18469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:50.377466 18469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:50.383035 18469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:37495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:50.384150 18469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:50.385365 18469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:50.395128 18474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:50.395128 18475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:50.395475 18469 server_base.cc:1034] running on GCE node
W20260809 03:25:50.395519 18477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:50.396080 18469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:50.397045 18469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:50.398270 18469 hybrid_clock.cc:648] HybridClock initialized: now 1786245950398245 us; error 35 us; skew 500 ppm
Aug 09 03:25:50 dist-test-slave-65mx krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:50.402132 18469 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:25:50.403191 18469 webserver.cc:460] Webserver started at http://127.9.46.195:32837/ using document root <none> and password file <none>
I20260809 03:25:50.403772 18469 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:50.403867 18469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:50.404115 18469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:25:50.406569 18469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "eb1094ee5b2d4f54a0b098d0007ae6c0"
format_stamp: "Formatted at 2026-08-09 03:25:50 on dist-test-slave-65mx"
server_key: "701730d3b52f07dd7b39ad46d4ad0577"
server_key_iv: "d80cf04d9b49287ef7d73fc0219e721d"
server_key_version: "encryptionkey@0"
I20260809 03:25:50.407171 18469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "eb1094ee5b2d4f54a0b098d0007ae6c0"
format_stamp: "Formatted at 2026-08-09 03:25:50 on dist-test-slave-65mx"
server_key: "701730d3b52f07dd7b39ad46d4ad0577"
server_key_iv: "d80cf04d9b49287ef7d73fc0219e721d"
server_key_version: "encryptionkey@0"
I20260809 03:25:50.411407 18469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 03:25:50.414144 18485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:50.415612 18469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:25:50.415737 18469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "eb1094ee5b2d4f54a0b098d0007ae6c0"
format_stamp: "Formatted at 2026-08-09 03:25:50 on dist-test-slave-65mx"
server_key: "701730d3b52f07dd7b39ad46d4ad0577"
server_key_iv: "d80cf04d9b49287ef7d73fc0219e721d"
server_key_version: "encryptionkey@0"
I20260809 03:25:50.415855 18469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:50.436198 18469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:50.439684 18469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:50.439903 18469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:50.440595 18469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:50.441556 18469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:50.441612 18469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:50.441681 18469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:50.441713 18469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:50.450917 18469 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:44219
I20260809 03:25:50.450986 18606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:44219 every 8 connection(s)
I20260809 03:25:50.451973 18469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:50.455359 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 18469
I20260809 03:25:50.455506 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:25:50.455794 9403 external_mini_cluster.cc:1428] Setting key 5a3d1af99f052df75113876cfe872f5d
Aug 09 03:25:50 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:25:50.465914 18156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:50.454063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:56007 (local address 127.9.46.254:37495)
0809 03:25:50.454234 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:50.454239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:50.455101 (+ 862us) server_negotiation.cc:407] Connection header received
0809 03:25:50.456620 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:50.456623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:50.456686 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:50.456787 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:50.458356 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:50.458887 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:50.460001 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:50.460223 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:50.460305 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:50.462781 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:50.462784 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:50.462786 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:50.462814 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:50.464076 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:50.464539 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:50.464594 (+ 55us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:50.464596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:50.464671 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:50.464976 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:50.464978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:50.464979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:50.465160 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:50.465237 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:50.465694 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 03:25:50.465790 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 03:25:50.466807 18609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:50.454482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:56007)
0809 03:25:50.454937 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:50.454981 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:50.456343 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:50.457002 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:50.457011 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:50.457556 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:50.458215 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:50.458240 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:50.458996 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:50.458998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:50.459888 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:50.459898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:50.460360 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:50.460362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:50.460505 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:50.461130 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:50.461146 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:50.462618 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:25:50.464185 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:50.464190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:50.464202 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:50.464455 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:50.464755 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:25:50.464757 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:25:50.464759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:25:50.464894 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:25:50.465251 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:25:50.465256 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:25:50.465491 (+ 235us) client_negotiation.cc:769] Sending connection context
0809 03:25:50.465653 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 03:25:50.465979 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":118,"threads_started":1}
I20260809 03:25:50.467995 18607 heartbeater.cc:347] Connected to a master server at 127.9.46.254:37495
I20260809 03:25:50.468259 18607 heartbeater.cc:464] Registering TS with master...
I20260809 03:25:50.468920 18607 heartbeater.cc:511] Master 127.9.46.254:37495 requested a full tablet report, sending...
I20260809 03:25:50.470136 18115 ts_manager.cc:194] Registered new tserver with Master: eb1094ee5b2d4f54a0b098d0007ae6c0 (127.9.46.195:44219)
I20260809 03:25:50.470662 18115 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:56007
I20260809 03:25:50.482093 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab.
Entry for principal oryx/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab.
WARNING: no policy specified for oryx/127.9.46.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab.
Entry for principal oryx/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab.
WARNING: no policy specified for oryx/127.9.46.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab.
Entry for principal oryx/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab.
I20260809 03:25:50.568228 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 18181
I20260809 03:25:50.577447 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 18327
I20260809 03:25:50.584822 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 18469
I20260809 03:25:50.593065 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 18080
I20260809 03:25:50.602169 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:37495
--webserver_interface=127.9.46.254
--webserver_port=42267
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:37495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:25:50.717605 18616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:50.717893 18616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:50.717935 18616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:50.722007 18616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:25:50.722080 18616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:50.722100 18616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:50.722118 18616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:25:50.722138 18616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:25:50.727448 18616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:37495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:37495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=42267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:50.728502 18616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:50.729683 18616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:50.738165 18621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:50.738375 18616 server_base.cc:1034] running on GCE node
W20260809 03:25:50.738209 18623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:50.738190 18625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:50.739190 18616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:50.740312 18616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:50.741518 18616 hybrid_clock.cc:648] HybridClock initialized: now 1786245950741495 us; error 36 us; skew 500 ppm
Aug 09 03:25:50 dist-test-slave-65mx krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:50.744477 18616 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:25:50.745398 18616 webserver.cc:460] Webserver started at http://127.9.46.254:42267/ using document root <none> and password file <none>
I20260809 03:25:50.745975 18616 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:50.746124 18616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:50.750110 18616 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:25:50.752406 18633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:50.753779 18616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:50.753935 18616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "297d565cdb9046349acb06413613902f"
format_stamp: "Formatted at 2026-08-09 03:25:49 on dist-test-slave-65mx"
server_key: "d72873cbc7b5084074a02d199bddd67f"
server_key_iv: "a3011780122c38736216fec62c8785fb"
server_key_version: "encryptionkey@0"
I20260809 03:25:50.754524 18616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:50.775542 18616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:50.779098 18616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:50.779311 18616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:50.787160 18616 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:37495
I20260809 03:25:50.787180 18689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:37495 every 8 connection(s)
I20260809 03:25:50.788295 18616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:25:50.790091 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 18616
I20260809 03:25:50.790552 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:38453
--local_ip_for_outbound_sockets=127.9.46.193
--tserver_master_addrs=127.9.46.254:37495
--webserver_port=41381
--webserver_interface=127.9.46.193
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.46.193
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:25:50.793607 18690 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:25:50.796749 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f: Bootstrap starting.
I20260809 03:25:50.819427 18690 log.cc:824] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:25:50.826496 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:25:50.826958 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f: Bootstrap complete.
I20260809 03:25:50.831807 18690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } }
I20260809 03:25:50.832276 18690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297d565cdb9046349acb06413613902f, State: Initialized, Role: FOLLOWER
I20260809 03:25:50.832939 18690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } }
I20260809 03:25:50.833110 18690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:25:50.833212 18690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:25:50.833340 18690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:25:50.835461 18690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } }
I20260809 03:25:50.835820 18690 leader_election.cc:302] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 297d565cdb9046349acb06413613902f; no voters:
I20260809 03:25:50.836220 18690 leader_election.cc:288] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:25:50.836320 18694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:25:50.836535 18694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [term 2 LEADER]: Becoming Leader. State: Replica: 297d565cdb9046349acb06413613902f, State: Running, Role: LEADER
I20260809 03:25:50.836871 18694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } }
I20260809 03:25:50.837616 18690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:25:50.838316 18695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "297d565cdb9046349acb06413613902f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } } }
I20260809 03:25:50.838446 18695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:50.839308 18700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:25:50.839412 18696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 297d565cdb9046349acb06413613902f. Latest consensus state: current_term: 2 leader_uuid: "297d565cdb9046349acb06413613902f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "297d565cdb9046349acb06413613902f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37495 } } }
I20260809 03:25:50.839623 18696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f [sys.catalog]: This master's current role is: LEADER
I20260809 03:25:50.842936 18700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:25:50.843657 18700 catalog_manager.cc:1259] Loaded cluster ID: 36640619ed3f4f03b2e72fe516878660
I20260809 03:25:50.843729 18700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:25:50.846335 18700 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:25:50.847050 18700 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 297d565cdb9046349acb06413613902f: Loaded TSK: 0
I20260809 03:25:50.848372 18700 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:25:50.915968 18692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:50.916292 18692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:50.916370 18692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:50.920655 18692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:50.920748 18692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:50.920840 18692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:25:50.926254 18692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.193.keytab
--principal=oryx/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:38453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=41381
--tserver_master_addrs=127.9.46.254:37495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:50.927373 18692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:50.928435 18692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:50.939294 18721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:50.939321 18718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:50.939404 18719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:50.940094 18692 server_base.cc:1034] running on GCE node
I20260809 03:25:50.940608 18692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:50.941308 18692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:50.942507 18692 hybrid_clock.cc:648] HybridClock initialized: now 1786245950942483 us; error 41 us; skew 500 ppm
Aug 09 03:25:50 dist-test-slave-65mx krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245950, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:50.945513 18692 init.cc:377] Logged in from keytab as oryx/127.9.46.193@KRBTEST.COM (short username oryx)
I20260809 03:25:50.946488 18692 webserver.cc:460] Webserver started at http://127.9.46.193:41381/ using document root <none> and password file <none>
I20260809 03:25:50.947080 18692 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:50.947158 18692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:50.951416 18692 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:25:50.953722 18728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:50.955089 18692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:25:50.955246 18692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "23b7fd781cb24e2bbe9c8d474b5e4b73"
format_stamp: "Formatted at 2026-08-09 03:25:49 on dist-test-slave-65mx"
server_key: "23231890e7879f1a7008d2a03134b66f"
server_key_iv: "d1eb6367a88a71be737b7f193fbb5e1e"
server_key_version: "encryptionkey@0"
I20260809 03:25:50.955837 18692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:50.989895 18692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:50.993237 18692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:50.993456 18692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:50.994088 18692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:50.994968 18692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:50.995044 18692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:50.995123 18692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:50.995165 18692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:51.005860 18692 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:38453
I20260809 03:25:51.005873 18848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:38453 every 8 connection(s)
I20260809 03:25:51.006994 18692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:25:51.009941 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 18692
I20260809 03:25:51.010449 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:43899
--local_ip_for_outbound_sockets=127.9.46.194
--tserver_master_addrs=127.9.46.254:37495
--webserver_port=45157
--webserver_interface=127.9.46.194
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.46.194
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:25:51.020998 18851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:51.009181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:47365 (local address 127.9.46.254:37495)
0809 03:25:51.009652 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:51.009704 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:51.010342 (+ 638us) server_negotiation.cc:407] Connection header received
0809 03:25:51.011516 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:51.011533 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:51.011913 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:51.012295 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:51.013693 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.014653 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.015790 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.016264 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.016442 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.020213 (+ 3771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:47365: BlockingRecv error: recv got EOF from 127.9.46.193:47365 (error 108)
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":129,"threads_started":1}
I20260809 03:25:51.021112 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:51.009317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:47365)
0809 03:25:51.010141 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:51.010205 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:51.011081 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:51.012730 (+ 1649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:51.012737 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:51.013085 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:51.013518 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.013532 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.014816 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.014819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.015662 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.015671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.016436 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.016440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.016642 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.017273 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:51.017289 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:51.020016 (+ 2727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":162,"threads_started":1}
W20260809 03:25:51.021600 18849 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:37495 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.46.254:37495: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:51.027523 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:51.022130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:38993)
0809 03:25:51.022249 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:51.022266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:51.022343 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:51.022563 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:51.022565 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:51.022696 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:51.022786 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.022790 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.023836 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.023839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.024390 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.024398 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.025028 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.025031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.025112 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.025629 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:51.025640 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:51.027427 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 03:25:51.027722 18851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:51.022215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:38993 (local address 127.9.46.254:37495)
0809 03:25:51.022333 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:51.022337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:51.022353 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:25:51.022401 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:51.022404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:51.022449 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:51.022537 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:51.022893 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.023706 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.024507 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.024921 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.025023 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.027599 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:38993: BlockingRecv error: recv got EOF from 127.9.46.193:38993 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:51.033424 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:51.028209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:37121)
0809 03:25:51.028318 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:51.028334 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:51.028409 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:51.028669 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:51.028671 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:51.028788 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:51.028854 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.028859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.029841 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.029843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.030338 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.030345 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.030926 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.030929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.031001 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.031524 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:51.031538 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:51.033304 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 03:25:51.033581 18851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:51.028312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:37121 (local address 127.9.46.254:37495)
0809 03:25:51.028415 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:51.028421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:51.028460 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:25:51.028495 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:51.028498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:51.028538 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:51.028637 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:51.028960 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.029738 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.030451 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.030807 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.030908 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.033436 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:37121: BlockingRecv error: recv got EOF from 127.9.46.193:37121 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 03:25:51.034041 18849 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:25:51.128432 18853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:51.128753 18853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:51.128829 18853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:51.132900 18853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:51.132977 18853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:51.133070 18853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:25:51.138708 18853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.194.keytab
--principal=oryx/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:43899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=45157
--tserver_master_addrs=127.9.46.254:37495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:51.139909 18853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:51.140982 18853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:51.150653 18859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:51.150638 18858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:51.150640 18862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:51.151093 18853 server_base.cc:1034] running on GCE node
I20260809 03:25:51.151510 18853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:51.152163 18853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:51.153359 18853 hybrid_clock.cc:648] HybridClock initialized: now 1786245951153325 us; error 49 us; skew 500 ppm
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245951, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:51.156081 18853 init.cc:377] Logged in from keytab as oryx/127.9.46.194@KRBTEST.COM (short username oryx)
I20260809 03:25:51.157006 18853 webserver.cc:460] Webserver started at http://127.9.46.194:45157/ using document root <none> and password file <none>
I20260809 03:25:51.157686 18853 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:51.157769 18853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:51.162081 18853 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:25:51.164220 18870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:51.165506 18853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:25:51.165642 18853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "4a3f1187e8ad454eb9ce0bfd7add57c7"
format_stamp: "Formatted at 2026-08-09 03:25:50 on dist-test-slave-65mx"
server_key: "cbe9ec38b8ccac540ff602ffe546df4c"
server_key_iv: "ca07fecd611b3354138f5a3e7708fcc8"
server_key_version: "encryptionkey@0"
I20260809 03:25:51.166196 18853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:51.193207 18853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:51.196367 18853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:51.196635 18853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:51.197299 18853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:51.198182 18853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:51.198230 18853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:51.198295 18853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:51.198329 18853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:51.207726 18853 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:43899
I20260809 03:25:51.207751 18989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:43899 every 8 connection(s)
I20260809 03:25:51.208834 18853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:25:51.218271 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 18853
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:51.218840 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:44219
--local_ip_for_outbound_sockets=127.9.46.195
--tserver_master_addrs=127.9.46.254:37495
--webserver_port=32837
--webserver_interface=127.9.46.195
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.46.195
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:25:51.220535 18851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:51.210793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:59093 (local address 127.9.46.254:37495)
0809 03:25:51.210970 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:51.210974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:51.211828 (+ 854us) server_negotiation.cc:407] Connection header received
0809 03:25:51.213267 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:51.213270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:51.213336 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:51.213445 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:51.214633 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.215168 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.216334 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.216638 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.216754 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.220410 (+ 3656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:59093: BlockingRecv error: recv got EOF from 127.9.46.194:59093 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 03:25:51.221321 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:51.211188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:59093)
0809 03:25:51.211670 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:51.211715 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:51.213039 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:51.213626 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:51.213634 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:51.213990 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:51.214471 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.214484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.215300 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.215303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.216198 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.216208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.216800 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.216804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.216989 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.217567 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:51.217584 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:51.220281 (+ 2697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":119,"threads_started":1}
W20260809 03:25:51.221828 18990 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:37495 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.46.254:37495: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:51.228632 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:51.222320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:36767)
0809 03:25:51.222511 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:51.222528 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:51.222633 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:51.222881 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:51.222887 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:51.223081 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:51.223228 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.223239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.223957 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.223962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.225036 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.225050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.225485 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.225492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.225632 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.226123 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:51.226144 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:51.228404 (+ 2260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104}
W20260809 03:25:51.228727 18851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:51.222403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:36767 (local address 127.9.46.254:37495)
0809 03:25:51.222518 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:51.222521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:51.222585 (+ 64us) server_negotiation.cc:407] Connection header received
0809 03:25:51.222725 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:51.222727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:51.222766 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:51.222833 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:51.223347 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.223829 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.225157 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.225364 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.225444 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.228592 (+ 3148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:36767: BlockingRecv error: recv got EOF from 127.9.46.194:36767 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:51.234855 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:51.229252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:35821)
0809 03:25:51.229397 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:51.229414 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:51.229506 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:51.229800 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:51.229802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:51.229939 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:51.230057 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.230061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.231193 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.231195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.231701 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.231708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.232306 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.232308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.232366 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.232885 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:51.232897 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:51.234724 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 03:25:51.235112 18990 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:25:51.235332 18851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:51.229391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:35821 (local address 127.9.46.254:37495)
0809 03:25:51.229553 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:51.229560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:51.229575 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:25:51.229619 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:51.229623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:51.229681 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:51.229777 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:51.230175 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.231056 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.231820 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.232192 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.232300 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.234884 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:35821: BlockingRecv error: recv got EOF from 127.9.46.194:35821 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 03:25:51.336885 18993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:25:51.337230 18993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:25:51.337322 18993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:25:51.342247 18993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:25:51.342308 18993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:25:51.342370 18993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:25:51.347927 18993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/krb5kdc/oryx_127.9.46.195.keytab
--principal=oryx/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:44219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=32837
--tserver_master_addrs=127.9.46.254:37495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:25:51.349038 18993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:25:51.350101 18993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:25:51.359959 18998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:25:51.359956 19001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:51.360234 18993 server_base.cc:1034] running on GCE node
W20260809 03:25:51.360009 18999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:25:51.360945 18993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:25:51.361639 18993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:25:51.362828 18993 hybrid_clock.cc:648] HybridClock initialized: now 1786245951362795 us; error 47 us; skew 500 ppm
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245951, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:25:51.365720 18993 init.cc:377] Logged in from keytab as oryx/127.9.46.195@KRBTEST.COM (short username oryx)
I20260809 03:25:51.366683 18993 webserver.cc:460] Webserver started at http://127.9.46.195:32837/ using document root <none> and password file <none>
I20260809 03:25:51.367264 18993 fs_manager.cc:359] Metadata directory not provided
I20260809 03:25:51.367378 18993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:25:51.371575 18993 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260809 03:25:51.373756 19008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:51.375020 18993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:25:51.375160 18993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "eb1094ee5b2d4f54a0b098d0007ae6c0"
format_stamp: "Formatted at 2026-08-09 03:25:50 on dist-test-slave-65mx"
server_key: "701730d3b52f07dd7b39ad46d4ad0577"
server_key_iv: "d80cf04d9b49287ef7d73fc0219e721d"
server_key_version: "encryptionkey@0"
I20260809 03:25:51.375722 18993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:25:51.390168 18993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:25:51.393088 18993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:25:51.393301 18993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:25:51.393952 18993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:25:51.394825 18993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:25:51.394899 18993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:51.394976 18993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:25:51.395020 18993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:25:51.404210 18993 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:44219
I20260809 03:25:51.404239 19130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:44219 every 8 connection(s)
I20260809 03:25:51.405272 18993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:25:51.407038 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 18993
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:25:51.419049 18851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:51.407586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:37703 (local address 127.9.46.254:37495)
0809 03:25:51.407759 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:51.407764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:51.408504 (+ 740us) server_negotiation.cc:407] Connection header received
0809 03:25:51.410508 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:51.410511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:51.410585 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:51.410689 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:51.412453 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.413309 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.414996 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.415345 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.415438 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.418926 (+ 3488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:37703: BlockingRecv error: recv got EOF from 127.9.46.195:37703 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 03:25:51.419697 19133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:51.407747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:37703)
0809 03:25:51.408329 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:51.408392 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:51.410245 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:51.411041 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:51.411060 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:51.411577 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:51.412271 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.412292 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.413491 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.413494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.414778 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.414789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.415487 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.415494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.415761 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.416338 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:51.416354 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:51.418821 (+ 2467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":138,"threads_started":1}
W20260809 03:25:51.420228 19131 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:37495 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.46.254:37495: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
I20260809 03:25:51.421288 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:51.408873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37624 (local address 127.9.46.254:37495)
0809 03:25:51.409192 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:51.409196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:51.410965 (+ 1769us) server_negotiation.cc:407] Connection header received
0809 03:25:51.411019 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:51.411025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:51.411107 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:51.411202 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:51.412272 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.413156 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.414833 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.415181 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.415279 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.417020 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:25:51.417026 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:25:51.417039 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:25:51.417067 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:25:51.418892 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:51.419431 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:51.419436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:51.419443 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:51.419578 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:25:51.419901 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:25:51.419904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:25:51.419906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:25:51.420353 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:25:51.420586 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 03:25:51.420916 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 03:25:51.421166 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":73,"threads_started":1}
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:51.425989 19133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:51.420778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:37461)
0809 03:25:51.420896 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:51.420911 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:51.420987 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:51.421352 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:51.421355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:51.421524 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:51.421634 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.421640 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.422397 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.422399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.422993 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.423003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.423550 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.423554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.423651 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.424129 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:51.424150 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:51.425901 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 03:25:51.426152 18851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:51.420958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:37461 (local address 127.9.46.254:37495)
0809 03:25:51.421073 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:51.421079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:51.421092 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:25:51.421195 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:51.421200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:51.421243 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:51.421336 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:51.421747 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.422284 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.423124 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.423421 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.423535 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.426025 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:37461: BlockingRecv error: recv got EOF from 127.9.46.195:37461 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:51 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:51.431133 19133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:51.426539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:57737)
0809 03:25:51.426690 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:51.426707 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:51.426799 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:51.426997 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:51.426999 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:51.427157 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:51.427227 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.427232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.427891 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.427893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.428410 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:51.428417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.428872 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.428874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:51.428931 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.429321 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:51.429331 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:51.430978 (+ 1647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 03:25:51.431200 18851 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:51.426681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:57737 (local address 127.9.46.254:37495)
0809 03:25:51.426827 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:51.426830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:51.426842 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:25:51.426870 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:51.426872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:51.426903 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:51.426965 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:51.427321 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.427787 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.428506 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:51.428773 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:51.428851 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:51.431095 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:57737: BlockingRecv error: recv got EOF from 127.9.46.195:57737 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260809 03:25:51.431411 19131 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 03:25:52 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:52 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:52.040604 19137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:52.034715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:58409)
0809 03:25:52.035005 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:52.035019 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:52.035115 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:52.035358 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:52.035360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:52.035541 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:52.035680 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:52.035686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.036453 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.036456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:52.037120 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:52.037129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.037562 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.037565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:52.037648 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:52.038255 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:52.038267 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:52.040460 (+ 2193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":91,"threads_started":1}
W20260809 03:25:52.040942 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:52.034795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:58409 (local address 127.9.46.254:37495)
0809 03:25:52.035085 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:52.035089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:52.035103 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:25:52.035206 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:52.035208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:52.035251 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:52.035319 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:52.035795 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.036347 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.037227 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.037465 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.037541 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:52.040672 (+ 3131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:58409: BlockingRecv error: recv got EOF from 127.9.46.193:58409 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":100,"threads_started":1}
Aug 09 03:25:52 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:52 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:52.241688 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:52.235841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:59513)
0809 03:25:52.236175 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:52.236189 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:52.236298 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:52.236571 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:52.236573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:52.236757 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:52.236856 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:52.236862 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.237648 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.237651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:52.238239 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:52.238248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.238733 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.238735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:52.238799 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:52.239384 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:52.239396 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:52.241478 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":140,"threads_started":1}
W20260809 03:25:52.241747 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:52.235908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:59513 (local address 127.9.46.254:37495)
0809 03:25:52.236065 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:52.236070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:52.236264 (+ 194us) server_negotiation.cc:407] Connection header received
0809 03:25:52.236377 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:52.236379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:52.236424 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:52.236502 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:52.236985 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.237548 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.238389 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.238623 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.238712 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:52.241628 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:59513: BlockingRecv error: recv got EOF from 127.9.46.194:59513 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 03:25:52 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:52 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:52.437860 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:52.432111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:42617)
0809 03:25:52.432395 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:52.432409 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:52.432513 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:52.432785 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:52.432787 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:52.432973 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:52.433097 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:52.433103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.433835 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.433837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:52.434419 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:52.434427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.434866 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.434868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:52.434929 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:52.435584 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:52.435600 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:52.437721 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":72,"threads_started":1}
W20260809 03:25:52.438048 19138 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:52.432194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:42617 (local address 127.9.46.254:37495)
0809 03:25:52.432378 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:52.432382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:52.432478 (+ 96us) server_negotiation.cc:407] Connection header received
0809 03:25:52.432631 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:52.432633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:52.432679 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:52.432748 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:52.433200 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.433725 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.434524 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:52.434762 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:52.434844 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:52.437915 (+ 3071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:42617: BlockingRecv error: recv got EOF from 127.9.46.195:42617 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 03:25:53 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:53 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:53.046708 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:53.041464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:49543)
0809 03:25:53.041776 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:53.041790 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:53.041885 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:53.042164 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:53.042166 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:53.042350 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:53.042448 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:53.042453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.043196 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.043198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:53.043741 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:53.043749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.044199 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.044201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:53.044262 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:53.044762 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:53.044774 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:53.046579 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":130,"threads_started":1}
W20260809 03:25:53.046903 19142 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:53.041529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:49543 (local address 127.9.46.254:37495)
0809 03:25:53.041807 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:53.041810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:53.041839 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:25:53.041973 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:53.041976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:53.042020 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:53.042123 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:53.042550 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.043099 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.043848 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.044096 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.044176 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:53.046725 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:49543: BlockingRecv error: recv got EOF from 127.9.46.193:49543 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":82,"threads_started":1}
Aug 09 03:25:53 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:53 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:53.251444 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:53.242686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:50275)
0809 03:25:53.242992 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:53.243008 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:53.243179 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:53.243476 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:53.243479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:53.243697 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:53.243798 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:53.243804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.244610 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.244613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:53.245188 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:53.245196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.245670 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.245674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:53.245763 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:53.249238 (+ 3475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:53.249277 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:53.251243 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":95,"threads_started":1}
W20260809 03:25:53.251507 19142 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:53.242752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:50275 (local address 127.9.46.254:37495)
0809 03:25:53.242946 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:53.242951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:53.243094 (+ 143us) server_negotiation.cc:407] Connection header received
0809 03:25:53.243290 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:53.243293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:53.243355 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:53.243451 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:53.243925 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.244458 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.245314 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.245544 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.245653 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:53.251392 (+ 5739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:50275: BlockingRecv error: recv got EOF from 127.9.46.194:50275 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 09 03:25:53 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:53 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:53.444339 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:53.438835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:39349)
0809 03:25:53.439146 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:53.439162 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:53.439272 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:53.439482 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:53.439484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:53.439665 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:53.439808 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:53.439818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.440649 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.440652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:53.441230 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:53.441239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.441709 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.441719 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:53.441795 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:53.442322 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:53.442337 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:53.444192 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":108,"threads_started":1}
W20260809 03:25:53.444509 19142 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:53.438925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:39349 (local address 127.9.46.254:37495)
0809 03:25:53.439113 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:53.439119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:53.439241 (+ 122us) server_negotiation.cc:407] Connection header received
0809 03:25:53.439354 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:53.439356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:53.439403 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:53.439473 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:53.439945 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.440480 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.441336 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:53.441559 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:53.441656 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:53.444346 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:39349: BlockingRecv error: recv got EOF from 127.9.46.195:39349 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 03:25:54 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:54 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:54.054332 19145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:54.047663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:45597)
0809 03:25:54.048026 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:54.048046 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:54.048194 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:54.048660 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:54.048665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:54.048960 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:54.049109 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:54.049119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.050207 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.050210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:54.050846 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:54.050855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.051319 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.051322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:54.051392 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:54.052007 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:54.052019 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:54.054194 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":83,"threads_started":1}
W20260809 03:25:54.054569 19146 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:54.047878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:45597 (local address 127.9.46.254:37495)
0809 03:25:54.048266 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:54.048277 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:54.048294 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:25:54.048340 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:54.048342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:54.048386 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:54.048532 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:54.049272 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.050090 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.050965 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.051205 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.051287 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:54.054399 (+ 3112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:45597: BlockingRecv error: recv got EOF from 127.9.46.193:45597 (error 108)
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":147,"threads_started":1}
Aug 09 03:25:54 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:54 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:54.258121 19148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:54.252390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:60149)
0809 03:25:54.252857 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:54.252871 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:54.253016 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:54.253376 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:54.253379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:54.253626 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:54.253764 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:54.253771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.254528 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.254531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:54.255086 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:54.255093 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.255529 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.255532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:54.255594 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:54.256081 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:54.256093 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:54.257976 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":119,"threads_started":1}
W20260809 03:25:54.258263 19146 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:54.252533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:60149 (local address 127.9.46.254:37495)
0809 03:25:54.252725 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:54.252729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:54.252959 (+ 230us) server_negotiation.cc:407] Connection header received
0809 03:25:54.253146 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:54.253149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:54.253223 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:54.253312 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:54.253888 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.254430 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.255196 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.255421 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.255499 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:54.258114 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:60149: BlockingRecv error: recv got EOF from 127.9.46.194:60149 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 03:25:54 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:54 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:54.450464 19149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:54.445216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:34939)
0809 03:25:54.445521 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:54.445536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:54.445636 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:54.445875 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:54.445877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:54.446065 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:54.446164 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:54.446170 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.446912 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.446915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:54.447481 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:54.447489 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.447902 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.447904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:54.447967 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:54.448471 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:54.448483 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:54.450341 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
W20260809 03:25:54.450613 19146 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:54.445304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:34939 (local address 127.9.46.254:37495)
0809 03:25:54.445435 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:54.445439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:54.445607 (+ 168us) server_negotiation.cc:407] Connection header received
0809 03:25:54.445716 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:54.445718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:54.445763 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:54.445841 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:54.446265 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.446793 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.447589 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:54.447802 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:54.447877 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:54.450491 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:34939: BlockingRecv error: recv got EOF from 127.9.46.195:34939 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 03:25:55 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:55 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:55.061138 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:55.055210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:47355)
0809 03:25:55.055525 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:55.055539 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:55.055644 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:55.055902 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:55.055904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:55.056099 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:55.056200 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:55.056205 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.057167 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.057170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:55.057784 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:55.057794 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.058270 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.058273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:55.058336 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:55.058885 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:55.058897 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:55.060940 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":80,"threads_started":1}
W20260809 03:25:55.061257 19151 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:55.055301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:47355 (local address 127.9.46.254:37495)
0809 03:25:55.055608 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:55.055611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:55.055626 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:25:55.055735 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:55.055738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:55.055783 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:55.055860 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:55.056305 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.057052 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.057936 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.058171 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.058248 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:55.061117 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:47355: BlockingRecv error: recv got EOF from 127.9.46.193:47355 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":88,"threads_started":1}
Aug 09 03:25:55 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:55 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:55.265621 19152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:55.259109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:45837)
0809 03:25:55.259445 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:55.259460 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:55.259609 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:55.259936 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:55.259939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:55.260185 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:55.260292 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:55.260299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.261217 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.261223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:55.261852 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:55.261861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.262395 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.262399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:55.262498 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:55.263056 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:55.263069 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:55.265409 (+ 2340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":96,"threads_started":1}
W20260809 03:25:55.265728 19151 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:55.259151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:45837 (local address 127.9.46.254:37495)
0809 03:25:55.259354 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:55.259359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:55.259546 (+ 187us) server_negotiation.cc:407] Connection header received
0809 03:25:55.259729 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:55.259732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:55.259790 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:55.259916 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:55.260427 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.261092 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.261998 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.262259 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.262387 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:55.265575 (+ 3188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:45837: BlockingRecv error: recv got EOF from 127.9.46.194:45837 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 03:25:55 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:55 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:55.458437 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:55.451525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:41495)
0809 03:25:55.451857 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:55.451882 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:55.452013 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:55.452399 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:55.452402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:55.452628 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:55.452731 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:55.452737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.454015 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.454019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:55.454814 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:55.454827 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.455560 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.455563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:55.455635 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:55.456287 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:55.456299 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:55.458314 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":112,"threads_started":1}
W20260809 03:25:55.458712 19151 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:55.451661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:41495 (local address 127.9.46.254:37495)
0809 03:25:55.451844 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:55.451852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:55.451981 (+ 129us) server_negotiation.cc:407] Connection header received
0809 03:25:55.452132 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:55.452138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:55.452208 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:55.452327 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:55.452871 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.453868 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.454941 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:55.455400 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:55.455530 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:55.458513 (+ 2983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:41495: BlockingRecv error: recv got EOF from 127.9.46.195:41495 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 03:25:56 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:56 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:56.067493 19154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:56.062026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:44649)
0809 03:25:56.062336 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:56.062349 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:56.062438 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:56.062696 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:56.062699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:56.062889 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:56.063011 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:56.063017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.063814 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.063816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:56.064409 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:56.064417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.064856 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.064858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:56.064922 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:56.065464 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:56.065477 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:56.067337 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":94,"threads_started":1}
W20260809 03:25:56.067795 19155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:56.062171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:44649 (local address 127.9.46.254:37495)
0809 03:25:56.062429 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:56.062433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:56.062447 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:25:56.062529 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:56.062531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:56.062574 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:56.062658 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:56.063114 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.063716 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.064511 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.064760 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.064837 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:56.067488 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:44649: BlockingRecv error: recv got EOF from 127.9.46.193:44649 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":94,"threads_started":1}
Aug 09 03:25:56 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:56 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:56.273247 19156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:56.266828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:54059)
0809 03:25:56.267212 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:56.267232 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:56.267404 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:56.267763 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:56.267766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:56.268109 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:56.268255 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:56.268264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.269217 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.269220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:56.269920 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:56.269930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.270556 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.270559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:56.270630 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:56.271197 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:56.271210 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:56.273127 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":105,"threads_started":1}
W20260809 03:25:56.273483 19155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:56.267048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:54059 (local address 127.9.46.254:37495)
0809 03:25:56.267216 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:56.267223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:56.267335 (+ 112us) server_negotiation.cc:407] Connection header received
0809 03:25:56.267549 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:56.267553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:56.267625 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:56.267733 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:56.268403 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.269089 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.270169 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.270437 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.270542 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:56.273331 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:54059: BlockingRecv error: recv got EOF from 127.9.46.194:54059 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 03:25:56 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:56 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:56.465221 19157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:56.459547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:48393)
0809 03:25:56.459896 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:56.459911 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:56.460044 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:56.460318 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:56.460320 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:56.460667 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:56.460769 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:56.460775 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.461513 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.461516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:56.462084 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:56.462096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.462525 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.462529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:56.462603 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:56.463235 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:56.463248 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:56.465103 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":125,"threads_started":1}
W20260809 03:25:56.465400 19155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:56.459630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:48393 (local address 127.9.46.254:37495)
0809 03:25:56.459808 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:56.459816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:56.460019 (+ 203us) server_negotiation.cc:407] Connection header received
0809 03:25:56.460146 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:56.460149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:56.460207 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:56.460277 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:56.460875 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.461402 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.462190 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:56.462403 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:56.462481 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:56.465238 (+ 2757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:48393: BlockingRecv error: recv got EOF from 127.9.46.195:48393 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 03:25:57 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:57 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:57.074676 19158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:57.068441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:40335)
0809 03:25:57.068791 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:57.068809 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:57.068935 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:57.069288 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:57.069291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:57.069546 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:57.069734 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:57.069744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.070680 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.070684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:57.071266 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:57.071275 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.071763 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.071765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:57.071836 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:57.072438 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:57.072451 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:57.074516 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":115,"threads_started":1}
W20260809 03:25:57.074968 19159 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:57.068481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:40335 (local address 127.9.46.254:37495)
0809 03:25:57.068803 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:57.068807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:57.068870 (+ 63us) server_negotiation.cc:407] Connection header received
0809 03:25:57.069049 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:57.069052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:57.069115 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:57.069258 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:57.069888 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.070559 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.071404 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.071639 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.071730 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:57.074710 (+ 2980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:40335: BlockingRecv error: recv got EOF from 127.9.46.193:40335 (error 108)
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":124,"threads_started":1}
Aug 09 03:25:57 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:57 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:57.279738 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:57.274121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:53427)
0809 03:25:57.274402 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:57.274416 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:57.274571 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:57.274850 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:57.274853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:57.275081 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:57.275178 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:57.275183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.275964 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.275967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:57.276563 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:57.276571 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.277052 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.277055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:57.277143 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:57.277680 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:57.277691 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:57.279616 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":97,"threads_started":1}
W20260809 03:25:57.279932 19159 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:57.274152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:53427 (local address 127.9.46.254:37495)
0809 03:25:57.274301 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:57.274307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:57.274507 (+ 200us) server_negotiation.cc:407] Connection header received
0809 03:25:57.274676 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:57.274679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:57.274734 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:57.274822 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:57.275300 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.275845 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.276681 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.276917 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.277044 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:57.279762 (+ 2718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:53427: BlockingRecv error: recv got EOF from 127.9.46.194:53427 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 03:25:57 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:57 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:57.471659 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:57.465995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:37229)
0809 03:25:57.466301 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:57.466315 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:57.466419 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:57.466723 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:57.466725 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:57.466915 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:57.467010 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:57.467016 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.467753 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.467756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:57.468320 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:57.468327 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.468820 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.468823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:57.468889 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:57.469480 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:57.469498 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:57.471528 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":95,"threads_started":1}
W20260809 03:25:57.471851 19159 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:57.466152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:37229 (local address 127.9.46.254:37495)
0809 03:25:57.466322 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:57.466325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:57.466370 (+ 45us) server_negotiation.cc:407] Connection header received
0809 03:25:57.466515 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:57.466517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:57.466572 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:57.466640 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:57.467129 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.467659 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.468429 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:57.468682 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:57.468782 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:57.471685 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:37229: BlockingRecv error: recv got EOF from 127.9.46.195:37229 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 03:25:58 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:58 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:58.081346 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:58.075528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:59335)
0809 03:25:58.075881 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:58.075895 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:58.075999 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:58.076261 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:58.076263 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:58.076452 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:58.076587 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:58.076594 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.077357 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.077359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:58.077910 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:58.077918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.078349 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.078352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:58.078417 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:58.079081 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:58.079094 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:58.081213 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":72,"threads_started":1}
W20260809 03:25:58.081578 19164 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:58.075604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:59335 (local address 127.9.46.254:37495)
0809 03:25:58.075889 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:58.075892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:58.075971 (+ 79us) server_negotiation.cc:407] Connection header received
0809 03:25:58.076098 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:58.076101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:58.076145 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:58.076222 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:58.076689 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.077257 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.078009 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.078245 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.078321 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:58.081406 (+ 3085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:59335: BlockingRecv error: recv got EOF from 127.9.46.193:59335 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":99,"threads_started":1}
Aug 09 03:25:58 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:58 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:58.291270 19166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:58.280636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:38517)
0809 03:25:58.281007 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:58.281022 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:58.281138 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:58.281388 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:58.281390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:58.281575 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:58.281684 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:58.281689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.282485 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.282488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:58.283356 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:58.283369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.283911 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.283914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:58.283980 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:58.289172 (+ 5192us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:58.289187 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:58.291143 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":73,"threads_started":1}
W20260809 03:25:58.291465 19164 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:58.280871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:38517 (local address 127.9.46.254:37495)
0809 03:25:58.281055 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:58.281059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:58.281076 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:25:58.281233 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:58.281235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:58.281281 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:58.281353 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:58.281786 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.282378 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.283492 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.283794 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.283879 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:58.291303 (+ 7424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:38517: BlockingRecv error: recv got EOF from 127.9.46.194:38517 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 03:25:58 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:58 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:58.477761 19167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:58.472481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:53547)
0809 03:25:58.472851 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:58.472865 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:58.472962 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:58.473199 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:58.473202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:58.473372 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:58.473467 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:58.473472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.474192 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.474195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:58.474750 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:58.474758 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.475179 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.475181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:58.475243 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:58.475731 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:58.475743 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:58.477646 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":132,"threads_started":1}
W20260809 03:25:58.477900 19164 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:58.472615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:53547 (local address 127.9.46.254:37495)
0809 03:25:58.472732 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:58.472736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:58.472932 (+ 196us) server_negotiation.cc:407] Connection header received
0809 03:25:58.473035 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:58.473037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:58.473085 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:58.473155 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:58.473559 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.474085 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.474852 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:58.475076 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:58.475159 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:58.477781 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:53547: BlockingRecv error: recv got EOF from 127.9.46.195:53547 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 03:25:59 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:59 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:59.087874 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:59.082221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:55139)
0809 03:25:59.082521 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:59.082537 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:59.082632 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:59.082911 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:59.082913 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:59.083142 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:59.083247 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:59.083252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.084035 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.084038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:59.084637 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:59.084646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.085111 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.085114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:59.085181 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:59.085833 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:59.085845 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:59.087751 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":115,"threads_started":1}
W20260809 03:25:59.088112 19169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:59.082385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:55139 (local address 127.9.46.254:37495)
0809 03:25:59.082610 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:59.082613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:59.082628 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:25:59.082736 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:59.082739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:59.082790 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:59.082867 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:59.083354 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.083895 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.084761 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.084988 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.085067 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:59.087956 (+ 2889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:55139: BlockingRecv error: recv got EOF from 127.9.46.193:55139 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":65,"threads_started":1}
Aug 09 03:25:59 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:59 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:59.298331 19170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:59.292245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:36997)
0809 03:25:59.292686 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:59.292704 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:59.292827 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:59.293128 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:59.293130 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:59.293317 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:59.293414 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:59.293419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.294363 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.294365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:59.294924 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:59.294932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.295418 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.295420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:59.295482 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:59.296075 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:59.296130 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:59.298210 (+ 2080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":102,"threads_started":1}
W20260809 03:25:59.298549 19169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:59.292290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:36997 (local address 127.9.46.254:37495)
0809 03:25:59.292479 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:59.292484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:59.292805 (+ 321us) server_negotiation.cc:407] Connection header received
0809 03:25:59.292953 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:59.292956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:59.293008 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:59.293076 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:59.293523 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.294263 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.295055 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.295309 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.295400 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:59.298374 (+ 2974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:36997: BlockingRecv error: recv got EOF from 127.9.46.194:36997 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 03:25:59 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:25:59 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:25:59.484299 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:25:59.478725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:52509)
0809 03:25:59.479029 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 03:25:59.479043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:25:59.479145 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:25:59.479363 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:25:59.479364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:25:59.479590 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:25:59.479693 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:59.479700 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.480655 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.480658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:59.481225 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:25:59.481233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.481674 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.481677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:25:59.481749 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:59.482275 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:25:59.482288 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:25:59.484177 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":126,"threads_started":1}
W20260809 03:25:59.484702 19169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:25:59.478803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:52509 (local address 127.9.46.254:37495)
0809 03:25:59.478948 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:25:59.478952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:25:59.479148 (+ 196us) server_negotiation.cc:407] Connection header received
0809 03:25:59.479241 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:25:59.479243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:25:59.479291 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:25:59.479351 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:25:59.479804 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.480503 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.481323 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:25:59.481532 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:25:59.481622 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:25:59.484429 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:52509: BlockingRecv error: recv got EOF from 127.9.46.195:52509 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 03:26:00 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:00 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:00.094230 19172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:00.088736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:46531)
0809 03:26:00.089004 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:00.089018 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:00.089125 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:00.089351 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:00.089354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:00.089531 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:00.089633 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:00.089638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.090406 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.090408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:00.090952 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:00.090960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.091386 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.091389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:00.091452 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:00.091968 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:00.091980 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:00.094044 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
W20260809 03:26:00.094308 19173 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:00.088855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:46531 (local address 127.9.46.254:37495)
0809 03:26:00.089137 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:00.089140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:00.089156 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:26:00.089191 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:00.089193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:00.089234 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:00.089315 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:00.089737 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.090307 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.091058 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.091286 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.091364 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:00.094181 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:46531: BlockingRecv error: recv got EOF from 127.9.46.193:46531 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":115,"threads_started":1}
Aug 09 03:26:00 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:00 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:00.305989 19174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:00.299397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:36207)
0809 03:26:00.299697 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:00.299717 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:00.299866 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:00.300213 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:00.300217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:00.300519 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:00.300733 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:00.300746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.301672 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.301675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:00.302357 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:00.302367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.302920 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.302923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:00.303054 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:00.303625 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:00.303638 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:00.305828 (+ 2190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":108,"threads_started":1}
W20260809 03:26:00.306138 19173 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:00.299555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:36207 (local address 127.9.46.254:37495)
0809 03:26:00.299741 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:00.299748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:00.299816 (+ 68us) server_negotiation.cc:407] Connection header received
0809 03:26:00.299998 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:00.300001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:00.300057 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:00.300156 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:00.300895 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.301546 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.302515 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.302750 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.302846 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:00.305998 (+ 3152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:36207: BlockingRecv error: recv got EOF from 127.9.46.194:36207 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 03:26:00 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:00 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:00.490682 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:00.485232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:51291)
0809 03:26:00.485512 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:00.485526 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:00.485627 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:00.485927 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:00.485929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:00.486117 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:00.486220 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:00.486226 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.486950 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.486952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:00.487511 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:00.487518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.487930 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.487933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:00.487995 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:00.488590 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:00.488601 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:00.490531 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":109,"threads_started":1}
W20260809 03:26:00.490861 19173 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:00.485271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:51291 (local address 127.9.46.254:37495)
0809 03:26:00.485424 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:00.485428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:00.485607 (+ 179us) server_negotiation.cc:407] Connection header received
0809 03:26:00.485743 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:00.485745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:00.485794 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:00.485886 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:00.486329 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.486851 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.487613 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:00.487831 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:00.487906 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:00.490693 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:51291: BlockingRecv error: recv got EOF from 127.9.46.195:51291 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 03:26:01 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:01 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:01.100972 19176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:01.095273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:58517)
0809 03:26:01.095585 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:01.095599 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:01.095690 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:01.095926 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:01.095928 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:01.096118 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:01.096217 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:01.096222 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.097148 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.097151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:01.097718 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:01.097727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.098169 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.098171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:01.098237 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:01.098746 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:01.098758 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:01.100849 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":113,"threads_started":1}
W20260809 03:26:01.101233 19177 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:01.095431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:58517 (local address 127.9.46.254:37495)
0809 03:26:01.095694 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:01.095697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:01.095712 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:26:01.095755 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:01.095757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:01.095797 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:01.095887 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:01.096357 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.097029 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.097827 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.098050 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.098128 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:01.101098 (+ 2970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:58517: BlockingRecv error: recv got EOF from 127.9.46.193:58517 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":110,"threads_started":1}
Aug 09 03:26:01 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:01 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:01.312316 19178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:01.306950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:46329)
0809 03:26:01.307263 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:01.307277 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:01.307373 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:01.307601 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:01.307603 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:01.307785 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:01.307882 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:01.307887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.308656 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.308659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:01.309234 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:01.309242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.309672 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.309674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:01.309734 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:01.310231 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:01.310243 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:01.312193 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":102,"threads_started":1}
W20260809 03:26:01.312503 19177 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:01.307046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:46329 (local address 127.9.46.254:37495)
0809 03:26:01.307217 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:01.307221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:01.307330 (+ 109us) server_negotiation.cc:407] Connection header received
0809 03:26:01.307454 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:01.307456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:01.307499 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:01.307558 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:01.307987 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.308503 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.309341 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.309558 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.309648 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:01.312343 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:46329: BlockingRecv error: recv got EOF from 127.9.46.194:46329 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 03:26:01 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:01 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:01.497517 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:01.491651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:51737)
0809 03:26:01.491972 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:01.491987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:01.492079 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:01.492429 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:01.492431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:01.492641 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:01.492739 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:01.492744 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.493523 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.493525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:01.494072 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:01.494079 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.494657 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.494660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:01.494728 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:01.495354 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:01.495371 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:01.497395 (+ 2024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":128,"threads_started":1}
W20260809 03:26:01.497876 19177 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:01.491843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:51737 (local address 127.9.46.254:37495)
0809 03:26:01.492059 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:01.492062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:01.492080 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:26:01.492247 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:01.492250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:01.492302 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:01.492386 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:01.492847 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.493419 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.494182 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:01.494531 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:01.494641 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:01.497634 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:51737: BlockingRecv error: recv got EOF from 127.9.46.195:51737 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 03:26:02 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:02 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:02.108103 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:02.102129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:37317)
0809 03:26:02.102480 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:02.102505 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:02.102609 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:02.102873 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:02.102875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:02.103078 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:02.103181 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:02.103186 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.103944 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.103946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:02.104626 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:02.104644 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.105141 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.105144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:02.105214 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:02.105921 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:02.105934 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:02.107913 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":93,"threads_started":1}
W20260809 03:26:02.108186 19181 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:02.102307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:37317 (local address 127.9.46.254:37495)
0809 03:26:02.102553 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:02.102556 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:02.102571 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:26:02.102696 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:02.102698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:02.102741 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:02.102821 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:02.103295 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.103838 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.104760 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.104986 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.105074 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:02.108059 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:37317: BlockingRecv error: recv got EOF from 127.9.46.193:37317 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":82,"threads_started":1}
Aug 09 03:26:02 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:02 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:02.319213 19182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:02.313221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:50243)
0809 03:26:02.313588 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:02.313607 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:02.313768 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:02.314074 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:02.314077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:02.314304 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:02.314402 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:02.314407 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.315201 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.315206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:02.315784 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:02.315792 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.316290 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.316294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:02.316389 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:02.317058 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:02.317071 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:02.319033 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":111,"threads_started":1}
W20260809 03:26:02.319321 19181 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:02.313322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:50243 (local address 127.9.46.254:37495)
0809 03:26:02.313564 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:02.313570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:02.313704 (+ 134us) server_negotiation.cc:407] Connection header received
0809 03:26:02.313877 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:02.313881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:02.313937 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:02.314038 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:02.314543 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.315084 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.315913 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.316160 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.316271 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:02.319200 (+ 2929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:50243: BlockingRecv error: recv got EOF from 127.9.46.194:50243 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
Aug 09 03:26:02 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:02 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:02.504920 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:02.498546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:47797)
0809 03:26:02.498847 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:02.498861 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:02.498972 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:02.499233 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:02.499234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:02.499423 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:02.499524 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:02.499530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.500288 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.500291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:02.500963 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:02.500973 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.501568 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.501570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:02.501636 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:02.502281 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:02.502297 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:02.504732 (+ 2435us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":123,"threads_started":1}
W20260809 03:26:02.504990 19181 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:02.498618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:47797 (local address 127.9.46.254:37495)
0809 03:26:02.498783 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:02.498789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:02.498958 (+ 169us) server_negotiation.cc:407] Connection header received
0809 03:26:02.499063 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:02.499065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:02.499121 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:02.499199 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:02.499630 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.500185 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.501169 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:02.501401 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:02.501560 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:02.504867 (+ 3307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:47797: BlockingRecv error: recv got EOF from 127.9.46.195:47797 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 03:26:03 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:03 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:03.115083 19184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:03.109113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:36067)
0809 03:26:03.109401 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:03.109414 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:03.109507 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:03.109780 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:03.109783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:03.109970 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:03.110070 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:03.110076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.110844 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.110846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:03.111400 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:03.111408 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.111894 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.111896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:03.111958 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:03.112599 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:03.112623 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:03.114916 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":102,"threads_started":1}
W20260809 03:26:03.115201 19185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:03.109232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:36067 (local address 127.9.46.254:37495)
0809 03:26:03.109487 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:03.109490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:03.109506 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:26:03.109608 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:03.109610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:03.109655 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:03.109748 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:03.110187 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.110732 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.111545 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.111783 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.111874 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:03.115065 (+ 3191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:36067: BlockingRecv error: recv got EOF from 127.9.46.193:36067 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
Aug 09 03:26:03 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:03 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:03.329252 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:03.320142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:40553)
0809 03:26:03.320436 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:03.320450 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:03.320581 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:03.320812 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:03.320814 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:03.321010 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:03.321110 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:03.321115 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.321962 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.321965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:03.322536 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:03.322545 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.323042 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.323045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:03.323107 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:03.326942 (+ 3835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:03.326957 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:03.329125 (+ 2168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":108,"threads_started":1}
W20260809 03:26:03.329493 19185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:03.320250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:40553 (local address 127.9.46.254:37495)
0809 03:26:03.320404 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:03.320413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:03.320510 (+ 97us) server_negotiation.cc:407] Connection header received
0809 03:26:03.320649 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:03.320651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:03.320694 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:03.320771 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:03.321250 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.321851 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.322674 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.322915 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.323007 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:03.329341 (+ 6334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:40553: BlockingRecv error: recv got EOF from 127.9.46.194:40553 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 03:26:03 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:03 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:03.511916 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:03.505825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:57509)
0809 03:26:03.506235 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:03.506249 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:03.506371 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:03.506649 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:03.506651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:03.506847 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:03.506958 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:03.506967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.507800 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.507803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:03.508464 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:03.508474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.509148 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.509151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:03.509237 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:03.509843 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:03.509854 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:03.511793 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":112,"threads_started":1}
W20260809 03:26:03.512120 19185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:03.505930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:57509 (local address 127.9.46.254:37495)
0809 03:26:03.506197 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:03.506201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:03.506350 (+ 149us) server_negotiation.cc:407] Connection header received
0809 03:26:03.506467 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:03.506469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:03.506518 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:03.506598 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:03.507152 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.507694 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.508673 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:03.509013 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:03.509143 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:03.511947 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:57509: BlockingRecv error: recv got EOF from 127.9.46.195:57509 (error 108)
Metrics: {"server-negotiator.queue_time_us":172}
Aug 09 03:26:04 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:04 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:04.122241 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:04.116132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:56273)
0809 03:26:04.116468 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:04.116483 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:04.116731 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:04.117047 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:04.117049 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:04.117249 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:04.117349 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:04.117354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.118104 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.118107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:04.118674 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:04.118682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.119106 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.119108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:04.119172 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:04.119708 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:04.119720 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:04.122065 (+ 2345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":77,"threads_started":1}
W20260809 03:26:04.122386 19189 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:04.116228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:56273 (local address 127.9.46.254:37495)
0809 03:26:04.116506 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:04.116510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:04.116564 (+ 54us) server_negotiation.cc:407] Connection header received
0809 03:26:04.116852 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:04.116854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:04.116904 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:04.116998 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:04.117458 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.117993 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.118781 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.119003 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.119105 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:04.122234 (+ 3129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:56273: BlockingRecv error: recv got EOF from 127.9.46.193:56273 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":87,"threads_started":1}
Aug 09 03:26:04 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:04 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:04.335950 19190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:04.330181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:57025)
0809 03:26:04.330585 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:04.330601 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:04.330707 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:04.331001 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:04.331003 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:04.331203 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:04.331311 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:04.331317 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.332104 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.332106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:04.332783 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:04.332793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.333258 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.333261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:04.333324 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:04.333835 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:04.333847 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:04.335781 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":91,"threads_started":1}
W20260809 03:26:04.336163 19189 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:04.330446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:57025 (local address 127.9.46.254:37495)
0809 03:26:04.330612 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:04.330615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:04.330668 (+ 53us) server_negotiation.cc:407] Connection header received
0809 03:26:04.330830 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:04.330832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:04.330883 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:04.330960 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:04.331444 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.331985 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.332904 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.333144 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.333237 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:04.335964 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:57025: BlockingRecv error: recv got EOF from 127.9.46.194:57025 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 03:26:04 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:04 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 03:26:04.519168 19189 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:04.513040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:39451 (local address 127.9.46.254:37495)
0809 03:26:04.513234 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:04.513239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:04.513382 (+ 143us) server_negotiation.cc:407] Connection header received
0809 03:26:04.513559 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:04.513562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:04.513620 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:04.513719 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:04.514187 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.514727 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.515562 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.515815 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.515954 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:04.519069 (+ 3115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:39451: BlockingRecv error: recv got EOF from 127.9.46.195:39451 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 03:26:04.519194 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:04.512891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:39451)
0809 03:26:04.513277 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:04.513293 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:04.513445 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:04.513748 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:04.513750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:04.513971 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:04.514066 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:04.514072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.514844 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.514849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:04.515438 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:04.515446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:04.515962 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:04.515965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:04.516071 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:04.516687 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:04.516700 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:04.518936 (+ 2236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":151,"threads_started":1}
Aug 09 03:26:05 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:05 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:05.128984 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:05.123222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:52493)
0809 03:26:05.123533 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:05.123547 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:05.123645 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:05.124031 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:05.124034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:05.124287 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:05.124389 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:05.124395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.125323 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.125326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:05.125942 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:05.125951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.126385 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.126388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:05.126449 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:05.126948 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:05.126960 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:05.128869 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":119,"threads_started":1}
W20260809 03:26:05.129210 19193 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:05.123302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:52493 (local address 127.9.46.254:37495)
0809 03:26:05.123650 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:05.123654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:05.123669 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:26:05.123728 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:05.123729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:05.123772 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:05.123862 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:05.124505 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.125223 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.126050 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.126283 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.126364 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:05.129007 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:52493: BlockingRecv error: recv got EOF from 127.9.46.193:52493 (error 108)
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":69,"threads_started":1}
Aug 09 03:26:05 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:05 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:05.343452 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:05.337203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:41615)
0809 03:26:05.337476 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:05.337498 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:05.337608 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:05.337881 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:05.337883 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:05.338187 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:05.338329 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:05.338338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.339346 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.339349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:05.340071 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:05.340088 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.340585 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.340590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:05.340675 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:05.341202 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:05.341213 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:05.343259 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
W20260809 03:26:05.343495 19193 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:05.337279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:41615 (local address 127.9.46.254:37495)
0809 03:26:05.337439 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:05.337443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:05.337551 (+ 108us) server_negotiation.cc:407] Connection header received
0809 03:26:05.337682 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:05.337685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:05.337729 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:05.337809 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:05.338448 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.339207 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.340216 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.340442 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.340532 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:05.343388 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:41615: BlockingRecv error: recv got EOF from 127.9.46.194:41615 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 03:26:05 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:05 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.195@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:05.525740 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:05.520176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.195:57607)
0809 03:26:05.520447 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:05.520460 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:05.520621 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:05.520940 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:05.520943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:05.521138 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:05.521240 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:05.521246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.522001 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.522005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:05.522602 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:05.522611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.523041 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.523044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:05.523106 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:05.523576 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:05.523588 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:05.525546 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
W20260809 03:26:05.525797 19193 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:05.520247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:57607 (local address 127.9.46.254:37495)
0809 03:26:05.520386 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:05.520390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:05.520584 (+ 194us) server_negotiation.cc:407] Connection header received
0809 03:26:05.520733 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:05.520735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:05.520813 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:05.520912 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:05.521351 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.521896 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.522718 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:05.522939 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:05.523018 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:05.525680 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.195:57607: BlockingRecv error: recv got EOF from 127.9.46.195:57607 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 03:26:06 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:06 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.193@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:06.136723 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:06.130064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.193:54463)
0809 03:26:06.130471 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:06.130490 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:06.130671 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:06.131052 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:06.131056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:06.131283 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:06.131390 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:06.131397 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.132300 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.132306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:06.133086 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:06.133097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.133685 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.133688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:06.133765 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:06.134404 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:06.134425 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:06.136605 (+ 2180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":104,"threads_started":1}
W20260809 03:26:06.136904 19197 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:06.130146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:54463 (local address 127.9.46.254:37495)
0809 03:26:06.130487 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:06.130493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:06.130603 (+ 110us) server_negotiation.cc:407] Connection header received
0809 03:26:06.130788 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:06.130793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:06.130854 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:06.130965 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:06.131536 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.132170 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.133223 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.133500 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.133654 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:06.136734 (+ 3080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.193:54463: BlockingRecv error: recv got EOF from 127.9.46.193:54463 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":71,"threads_started":1}
Aug 09 03:26:06 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 03:26:06 dist-test-slave-65mx krb5kdc[18066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.46.194@KRBTEST.COM for oryx/127.9.46.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 03:26:06.349967 19198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:06.344653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37495 (local address 127.9.46.194:40445)
0809 03:26:06.344925 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:06.344939 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:06.345030 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:06.345246 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:06.345248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:06.345500 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:06.345623 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:06.345629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.346350 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.346353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:06.346905 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:06.346912 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.347340 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.347343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:06.347407 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:06.347901 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:06.347913 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:06.349843 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.46.254:37495: Server oryx/127.9.46.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":106,"threads_started":1}
W20260809 03:26:06.350183 19197 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:06.344684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:40445 (local address 127.9.46.254:37495)
0809 03:26:06.344828 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:06.344831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:06.345009 (+ 178us) server_negotiation.cc:407] Connection header received
0809 03:26:06.345094 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:06.345096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:06.345139 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:06.345207 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:06.345730 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.346252 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.347014 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.347236 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.347319 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:06.350040 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.46.194:40445: BlockingRecv error: recv got EOF from 127.9.46.194:40445 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:26:06.412304 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 18692
I20260809 03:26:06.420989 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 18853
I20260809 03:26:06.428928 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 18993
I20260809 03:26:06.435554 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 18616
2026-08-09T03:26:06Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18218 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T03:26:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:26:06Z Disabled control of system clock
I20260809 03:26:06.469770 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34793
--webserver_interface=127.9.46.254
--webserver_port=0
--builtin_ntp_servers=127.9.46.212:35131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:34793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 03:26:06.582708 19203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:06.582978 19203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:06.583012 19203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:06.587013 19203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:26:06.587069 19203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:06.587087 19203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:06.587105 19203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:26:06.587119 19203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:26:06.591773 19203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:34793
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:06.592805 19203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:06.594117 19203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:06.602897 19209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:06.602900 19210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:06.602952 19213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:06.603425 19203 server_base.cc:1034] running on GCE node
I20260809 03:26:06.604030 19203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:06.605288 19203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:06.606483 19203 hybrid_clock.cc:648] HybridClock initialized: now 1786245966606467 us; error 30 us; skew 500 ppm
I20260809 03:26:06.608345 19203 webserver.cc:460] Webserver started at http://127.9.46.254:33309/ using document root <none> and password file <none>
I20260809 03:26:06.608994 19203 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:06.609086 19203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:06.609316 19203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:06.611382 19203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "d1b2a64c39ee4f21b211e478df19ee4a"
format_stamp: "Formatted at 2026-08-09 03:26:06 on dist-test-slave-65mx"
server_key: "8790575c02f7e1dcc144df269320e190"
server_key_iv: "66a1faf10b00fe3ec44f7e0b3ad6870c"
server_key_version: "encryptionkey@0"
I20260809 03:26:06.611965 19203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "d1b2a64c39ee4f21b211e478df19ee4a"
format_stamp: "Formatted at 2026-08-09 03:26:06 on dist-test-slave-65mx"
server_key: "8790575c02f7e1dcc144df269320e190"
server_key_iv: "66a1faf10b00fe3ec44f7e0b3ad6870c"
server_key_version: "encryptionkey@0"
I20260809 03:26:06.615897 19203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 03:26:06.618291 19220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:06.619594 19203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:26:06.619765 19203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "d1b2a64c39ee4f21b211e478df19ee4a"
format_stamp: "Formatted at 2026-08-09 03:26:06 on dist-test-slave-65mx"
server_key: "8790575c02f7e1dcc144df269320e190"
server_key_iv: "66a1faf10b00fe3ec44f7e0b3ad6870c"
server_key_version: "encryptionkey@0"
I20260809 03:26:06.619894 19203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:06.632344 19203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:06.633131 19203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:06.633337 19203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:06.640810 19203 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:34793
I20260809 03:26:06.640825 19275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:34793 every 8 connection(s)
I20260809 03:26:06.641755 19203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:26:06.644510 19276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:26:06.645797 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 19203
I20260809 03:26:06.645931 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:26:06.646294 9403 external_mini_cluster.cc:1428] Setting key adba7d7628ddcbf6eb6ef50cb90acbba
I20260809 03:26:06.650835 19276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a: Bootstrap starting.
I20260809 03:26:06.653110 19276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a: Neither blocks nor log segments found. Creating new log.
I20260809 03:26:06.653858 19276 log.cc:824] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:26:06.655956 19276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a: No bootstrap required, opened a new log
I20260809 03:26:06.656665 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:06.647825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60196 (local address 127.9.46.254:34793)
0809 03:26:06.648471 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:06.648487 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:06.648525 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:26:06.649352 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:06.649371 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:06.649776 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:06.650138 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:06.650607 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.651706 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.652788 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.653373 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.653561 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:06.654568 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:06.654577 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:06.654598 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:26:06.655053 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:06.655111 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:26:06.655133 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:06.655214 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:06.655408 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 03:26:06.655717 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":119,"threads_started":1}
I20260809 03:26:06.659925 19276 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b2a64c39ee4f21b211e478df19ee4a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34793 } }
I20260809 03:26:06.660167 19276 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:26:06.660249 19276 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b2a64c39ee4f21b211e478df19ee4a, State: Initialized, Role: FOLLOWER
I20260809 03:26:06.660784 19276 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b2a64c39ee4f21b211e478df19ee4a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34793 } }
I20260809 03:26:06.660936 19276 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:26:06.661027 19276 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:26:06.661125 19276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:26:06.662257 19276 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b2a64c39ee4f21b211e478df19ee4a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34793 } }
I20260809 03:26:06.662608 19276 leader_election.cc:302] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1b2a64c39ee4f21b211e478df19ee4a; no voters:
I20260809 03:26:06.662979 19276 leader_election.cc:288] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:26:06.663138 19281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:26:06.663400 19281 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [term 1 LEADER]: Becoming Leader. State: Replica: d1b2a64c39ee4f21b211e478df19ee4a, State: Running, Role: LEADER
I20260809 03:26:06.663763 19281 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b2a64c39ee4f21b211e478df19ee4a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34793 } }
I20260809 03:26:06.664292 19276 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:26:06.665593 19282 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1b2a64c39ee4f21b211e478df19ee4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b2a64c39ee4f21b211e478df19ee4a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34793 } } }
I20260809 03:26:06.665586 19283 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1b2a64c39ee4f21b211e478df19ee4a. Latest consensus state: current_term: 1 leader_uuid: "d1b2a64c39ee4f21b211e478df19ee4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b2a64c39ee4f21b211e478df19ee4a" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34793 } } }
I20260809 03:26:06.665745 19283 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [sys.catalog]: This master's current role is: LEADER
I20260809 03:26:06.665745 19282 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a [sys.catalog]: This master's current role is: LEADER
I20260809 03:26:06.666477 19290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:26:06.669337 19290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:26:06.674805 19290 catalog_manager.cc:1347] Generated new cluster ID: 9c180646a3a34d6bb52d9df622b1c5ae
I20260809 03:26:06.674871 19290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:26:06.686429 19290 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:26:06.687283 19290 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:26:06.695364 19290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1b2a64c39ee4f21b211e478df19ee4a: Generated new TSK 0
I20260809 03:26:06.696094 19290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:26:06.702073 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34793
--builtin_ntp_servers=127.9.46.212:35131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 03:26:06.815593 19302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:06.815876 19302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:06.815951 19302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:06.820010 19302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:06.820092 19302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:06.820183 19302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:26:06.824980 19302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:06.826006 19302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:06.827351 19302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:06.836634 19314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:06.836634 19312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:06.837003 19302 server_base.cc:1034] running on GCE node
W20260809 03:26:06.837224 19311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:06.837639 19302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:06.838243 19302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:06.839430 19302 hybrid_clock.cc:648] HybridClock initialized: now 1786245966839413 us; error 34 us; skew 500 ppm
I20260809 03:26:06.841197 19302 webserver.cc:460] Webserver started at http://127.9.46.193:37607/ using document root <none> and password file <none>
I20260809 03:26:06.841768 19302 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:06.841861 19302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:06.842105 19302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:06.844101 19302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "0f695703dd9040cc9ebc929336a0f143"
format_stamp: "Formatted at 2026-08-09 03:26:06 on dist-test-slave-65mx"
server_key: "5149e28fff14854af99863d0475ded46"
server_key_iv: "ab2f2d35bfe744f38ac16267e0926ab6"
server_key_version: "encryptionkey@0"
I20260809 03:26:06.844691 19302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "0f695703dd9040cc9ebc929336a0f143"
format_stamp: "Formatted at 2026-08-09 03:26:06 on dist-test-slave-65mx"
server_key: "5149e28fff14854af99863d0475ded46"
server_key_iv: "ab2f2d35bfe744f38ac16267e0926ab6"
server_key_version: "encryptionkey@0"
I20260809 03:26:06.848286 19302 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:26:06.850988 19320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:06.852224 19302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 03:26:06.852358 19302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "0f695703dd9040cc9ebc929336a0f143"
format_stamp: "Formatted at 2026-08-09 03:26:06 on dist-test-slave-65mx"
server_key: "5149e28fff14854af99863d0475ded46"
server_key_iv: "ab2f2d35bfe744f38ac16267e0926ab6"
server_key_version: "encryptionkey@0"
I20260809 03:26:06.852478 19302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:06.870822 19302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:06.871537 19302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:06.871742 19302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:06.872344 19302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:06.873335 19302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:06.873407 19302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:06.873481 19302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:06.873522 19302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:06.882545 19302 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:46007
I20260809 03:26:06.882575 19442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:46007 every 8 connection(s)
I20260809 03:26:06.883478 19302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:26:06.889340 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 19302
I20260809 03:26:06.889434 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:26:06.889770 9403 external_mini_cluster.cc:1428] Setting key 7b63c8a5d53eaf60d3b249fa6d77c76c
I20260809 03:26:06.891835 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34793
--builtin_ntp_servers=127.9.46.212:35131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 03:26:06.893108 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:06.885363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:55737 (local address 127.9.46.254:34793)
0809 03:26:06.885545 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:06.885550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:06.886607 (+ 1057us) server_negotiation.cc:407] Connection header received
0809 03:26:06.887912 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:06.887915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:06.887962 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:06.888037 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:06.888989 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.889618 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.891022 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.891307 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.891390 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:06.892650 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:06.892652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:06.892654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:26:06.892666 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:06.892685 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:26:06.892693 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:06.892747 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:06.892930 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 03:26:06.892985 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 03:26:06.894152 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:06.885823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34793 (local address 127.9.46.193:55737)
0809 03:26:06.886462 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:06.886502 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:06.887736 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:06.888193 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:06.888199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:06.888274 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:06.888831 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:06.888844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.889747 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.889750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:06.890889 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:06.890902 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:06.891462 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:06.891467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:06.891684 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:06.892313 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:26:06.892317 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:06.892371 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:26:06.892454 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:06.892775 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:06.892787 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:06.892796 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 03:26:06.892923 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 03:26:06.893234 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":137,"threads_started":1}
I20260809 03:26:06.895395 19443 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34793
I20260809 03:26:06.895614 19443 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:06.896095 19443 heartbeater.cc:511] Master 127.9.46.254:34793 requested a full tablet report, sending...
I20260809 03:26:06.897742 19237 ts_manager.cc:194] Registered new tserver with Master: 0f695703dd9040cc9ebc929336a0f143 (127.9.46.193:46007)
I20260809 03:26:06.899471 19237 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.193:55737
W20260809 03:26:07.004609 19447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:07.004851 19447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:07.004889 19447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:07.008774 19447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:07.008819 19447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:07.008921 19447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:26:07.013422 19447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:07.014324 19447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:07.015535 19447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:07.024683 19455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:07.024675 19452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:07.024989 19447 server_base.cc:1034] running on GCE node
W20260809 03:26:07.024675 19453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:07.025528 19447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:07.026160 19447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:07.027321 19447 hybrid_clock.cc:648] HybridClock initialized: now 1786245967027308 us; error 25 us; skew 500 ppm
I20260809 03:26:07.029170 19447 webserver.cc:460] Webserver started at http://127.9.46.194:36347/ using document root <none> and password file <none>
I20260809 03:26:07.029738 19447 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:07.029799 19447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:07.030014 19447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:07.032114 19447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "2464395e753a428289473df8e79d0009"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "b20cc7bdbec15d9be57e0ed3797f0776"
server_key_iv: "d06e214720f3e131a008c92f15bc33c0"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.032809 19447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "2464395e753a428289473df8e79d0009"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "b20cc7bdbec15d9be57e0ed3797f0776"
server_key_iv: "d06e214720f3e131a008c92f15bc33c0"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.036461 19447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:26:07.038932 19462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.040124 19447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:26:07.040261 19447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "2464395e753a428289473df8e79d0009"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "b20cc7bdbec15d9be57e0ed3797f0776"
server_key_iv: "d06e214720f3e131a008c92f15bc33c0"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.040374 19447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:07.063906 19447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:07.064761 19447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:07.064980 19447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:07.065814 19447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:07.067118 19447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:07.067200 19447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.067274 19447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:07.067310 19447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.076596 19447 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:41597
I20260809 03:26:07.076622 19583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:41597 every 8 connection(s)
I20260809 03:26:07.077538 19447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:26:07.079553 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 19447
I20260809 03:26:07.079690 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:26:07.079996 9403 external_mini_cluster.cc:1428] Setting key 9826ed9794eb77b1cf5424f953552d5c
I20260809 03:26:07.082329 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34793
--builtin_ntp_servers=127.9.46.212:35131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 03:26:07.088604 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:07.079409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:34177 (local address 127.9.46.254:34793)
0809 03:26:07.079588 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:07.079592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:07.080881 (+ 1289us) server_negotiation.cc:407] Connection header received
0809 03:26:07.082675 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:07.082678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:07.082724 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:07.082794 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:07.083972 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:07.084493 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:07.086315 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:07.086534 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:07.086627 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:07.088053 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:07.088056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:07.088058 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:26:07.088071 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:07.088091 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:26:07.088099 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:07.088153 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:07.088362 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 03:26:07.088422 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:26:07.089923 19586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:07.079867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34793 (local address 127.9.46.194:34177)
0809 03:26:07.080714 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:07.080771 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:07.082398 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:07.082980 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:07.082991 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:07.083115 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:07.083817 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:07.083838 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:07.084676 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:07.084681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:07.086177 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:07.086192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:07.086709 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:07.086716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:07.086953 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:07.087699 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:26:07.087703 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:07.087765 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:26:07.087867 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:07.088190 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:07.088203 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:07.088213 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 03:26:07.088363 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 03:26:07.088747 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":179,"threads_started":1}
I20260809 03:26:07.091266 19584 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34793
I20260809 03:26:07.091532 19584 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:07.092088 19584 heartbeater.cc:511] Master 127.9.46.254:34793 requested a full tablet report, sending...
I20260809 03:26:07.093226 19237 ts_manager.cc:194] Registered new tserver with Master: 2464395e753a428289473df8e79d0009 (127.9.46.194:41597)
I20260809 03:26:07.093816 19237 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.194:34177
W20260809 03:26:07.196221 19587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:07.196485 19587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:07.196539 19587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:07.200676 19587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:07.200729 19587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:07.200811 19587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:26:07.205550 19587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:07.206656 19587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:07.208016 19587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:07.217368 19594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:07.217566 19587 server_base.cc:1034] running on GCE node
W20260809 03:26:07.217393 19596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:07.217381 19593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:07.218060 19587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:07.218710 19587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:07.219895 19587 hybrid_clock.cc:648] HybridClock initialized: now 1786245967219875 us; error 31 us; skew 500 ppm
I20260809 03:26:07.221812 19587 webserver.cc:460] Webserver started at http://127.9.46.195:44929/ using document root <none> and password file <none>
I20260809 03:26:07.222375 19587 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:07.222470 19587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:07.222711 19587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:07.224856 19587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "83e9fb66096a419d9622cbf95f1767f1"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "433b3c6c24d2ae21ae52504abc88ecbe"
server_key_iv: "d7af7814f6bacc7b704e5232a171c0a2"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.225442 19587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "83e9fb66096a419d9622cbf95f1767f1"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "433b3c6c24d2ae21ae52504abc88ecbe"
server_key_iv: "d7af7814f6bacc7b704e5232a171c0a2"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.229137 19587 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:26:07.231613 19602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.232880 19587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:26:07.233013 19587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "83e9fb66096a419d9622cbf95f1767f1"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "433b3c6c24d2ae21ae52504abc88ecbe"
server_key_iv: "d7af7814f6bacc7b704e5232a171c0a2"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.233136 19587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:07.258715 19587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:07.259397 19587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:07.259554 19587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:07.260098 19587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:07.261035 19587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:07.261083 19587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.261124 19587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:07.261140 19587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.270357 19587 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:42427
I20260809 03:26:07.270421 19722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:42427 every 8 connection(s)
I20260809 03:26:07.271306 19587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:26:07.280479 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 19587
I20260809 03:26:07.280651 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:26:07.281008 9403 external_mini_cluster.cc:1428] Setting key 691116460ef8840b84787a6096a2c694
I20260809 03:26:07.281054 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:07.273192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:58655 (local address 127.9.46.254:34793)
0809 03:26:07.273340 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:07.273344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:07.274412 (+ 1068us) server_negotiation.cc:407] Connection header received
0809 03:26:07.275834 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:07.275837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:07.275895 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:07.276010 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:07.277241 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:07.277781 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:07.278947 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:07.279185 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:07.279303 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:07.280437 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:07.280440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:07.280443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:26:07.280458 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:07.280486 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:26:07.280494 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:07.280599 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:07.280820 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 03:26:07.280898 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 03:26:07.282011 19725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:07.273609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34793 (local address 127.9.46.195:58655)
0809 03:26:07.274257 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:07.274297 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:07.275622 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:07.276233 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:07.276246 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:07.276349 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:07.277090 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:07.277103 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:07.277916 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:07.277919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:07.278814 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:07.278823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:07.279349 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:07.279353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:07.279537 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:07.280173 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:26:07.280175 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:07.280215 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:26:07.280279 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:07.280610 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:07.280623 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:07.280632 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 03:26:07.280796 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 03:26:07.281029 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":126,"threads_started":1}
I20260809 03:26:07.283143 19723 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34793
I20260809 03:26:07.283401 19723 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:07.283923 19723 heartbeater.cc:511] Master 127.9.46.254:34793 requested a full tablet report, sending...
I20260809 03:26:07.284956 19237 ts_manager.cc:194] Registered new tserver with Master: 83e9fb66096a419d9622cbf95f1767f1 (127.9.46.195:42427)
I20260809 03:26:07.285537 19237 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.195:58655
I20260809 03:26:07.295425 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 03:26:07.298650 19279 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:07.297499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.9.46.254:34793)
0809 03:26:07.297666 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:07.297670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:07.297849 (+ 179us) server_negotiation.cc:407] Connection header received
0809 03:26:07.298021 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:07.298025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:07.298069 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:07.298158 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:07.298474 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60208: BlockingRecv error: recv got EOF from 127.0.0.1:60208 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:26:07.298738 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 19302
I20260809 03:26:07.308907 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 19447
I20260809 03:26:07.315496 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 19587
I20260809 03:26:07.323788 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 19203
2026-08-09T03:26:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (886 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T03:26:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:26:07Z Disabled control of system clock
I20260809 03:26:07.356410 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:33621
--webserver_interface=127.9.46.254
--webserver_port=0
--builtin_ntp_servers=127.9.46.212:34555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:33621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 03:26:07.469643 19735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:07.469923 19735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:07.469961 19735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:07.474152 19735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:26:07.474213 19735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:07.474229 19735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:07.474247 19735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:26:07.474303 19735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:26:07.479110 19735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:33621
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:33621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:07.480230 19735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:07.481724 19735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:07.490671 19742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:07.490712 19745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:07.490671 19741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:07.491030 19735 server_base.cc:1034] running on GCE node
I20260809 03:26:07.491644 19735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:07.492858 19735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:07.494021 19735 hybrid_clock.cc:648] HybridClock initialized: now 1786245967494011 us; error 27 us; skew 500 ppm
I20260809 03:26:07.496021 19735 webserver.cc:460] Webserver started at http://127.9.46.254:36581/ using document root <none> and password file <none>
I20260809 03:26:07.496692 19735 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:07.496765 19735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:07.497012 19735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:07.499133 19735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "7cde6dea2da147d3b1a3449dc238257c"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "5228d8d194133edd70917121f9f14fcd"
server_key_iv: "c6539437f26e9bba286bd1bf11dbc469"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.499713 19735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "7cde6dea2da147d3b1a3449dc238257c"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "5228d8d194133edd70917121f9f14fcd"
server_key_iv: "c6539437f26e9bba286bd1bf11dbc469"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.503751 19735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 03:26:07.506374 19752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.507841 19735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 03:26:07.507982 19735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "7cde6dea2da147d3b1a3449dc238257c"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "5228d8d194133edd70917121f9f14fcd"
server_key_iv: "c6539437f26e9bba286bd1bf11dbc469"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.508105 19735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:07.512692 19735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:07.513286 19735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:07.513473 19735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:07.520524 19735 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:33621
I20260809 03:26:07.520534 19807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:33621 every 8 connection(s)
I20260809 03:26:07.521553 19735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:26:07.522393 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 19735
I20260809 03:26:07.522528 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:26:07.522817 9403 external_mini_cluster.cc:1428] Setting key 7802f2fbbe3914f75abb5b0bd3db65e7
I20260809 03:26:07.524752 19808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:26:07.529366 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:07.524220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42906 (local address 127.9.46.254:33621)
0809 03:26:07.525383 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:07.525408 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:07.525450 (+ 42us) server_negotiation.cc:407] Connection header received
0809 03:26:07.526335 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:07.526358 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:07.526787 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:07.527146 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:07.527868 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:07.527873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:07.527886 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:26:07.527898 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:07.527967 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:26:07.527991 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:07.528130 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:07.528198 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 03:26:07.528437 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":633,"threads_started":1}
I20260809 03:26:07.531790 19808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c: Bootstrap starting.
I20260809 03:26:07.533885 19808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c: Neither blocks nor log segments found. Creating new log.
I20260809 03:26:07.534572 19808 log.cc:824] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:26:07.536505 19808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c: No bootstrap required, opened a new log
I20260809 03:26:07.540074 19808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde6dea2da147d3b1a3449dc238257c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33621 } }
I20260809 03:26:07.540289 19808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:26:07.540324 19808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cde6dea2da147d3b1a3449dc238257c, State: Initialized, Role: FOLLOWER
I20260809 03:26:07.540732 19808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde6dea2da147d3b1a3449dc238257c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33621 } }
I20260809 03:26:07.540855 19808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:26:07.540901 19808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:26:07.540978 19808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:26:07.541811 19808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde6dea2da147d3b1a3449dc238257c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33621 } }
I20260809 03:26:07.542174 19808 leader_election.cc:302] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cde6dea2da147d3b1a3449dc238257c; no voters:
I20260809 03:26:07.542527 19808 leader_election.cc:288] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:26:07.542660 19813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:26:07.542924 19813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [term 1 LEADER]: Becoming Leader. State: Replica: 7cde6dea2da147d3b1a3449dc238257c, State: Running, Role: LEADER
I20260809 03:26:07.543224 19813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde6dea2da147d3b1a3449dc238257c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33621 } }
I20260809 03:26:07.543718 19808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:26:07.544837 19815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cde6dea2da147d3b1a3449dc238257c. Latest consensus state: current_term: 1 leader_uuid: "7cde6dea2da147d3b1a3449dc238257c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde6dea2da147d3b1a3449dc238257c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33621 } } }
I20260809 03:26:07.544961 19815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [sys.catalog]: This master's current role is: LEADER
I20260809 03:26:07.545255 19814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cde6dea2da147d3b1a3449dc238257c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cde6dea2da147d3b1a3449dc238257c" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33621 } } }
I20260809 03:26:07.545347 19814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c [sys.catalog]: This master's current role is: LEADER
I20260809 03:26:07.545447 19822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:26:07.548691 19822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:26:07.554304 19822 catalog_manager.cc:1347] Generated new cluster ID: 0aedc54c0feb4295972a0f4270fbaeea
I20260809 03:26:07.554387 19822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:26:07.568519 19822 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:26:07.568895 19822 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:26:07.579530 19822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cde6dea2da147d3b1a3449dc238257c: Generated new TSK 0
I20260809 03:26:07.580327 19822 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:26:07.584417 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33621
--builtin_ntp_servers=127.9.46.212:34555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 03:26:07.695888 19838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:07.696187 19838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:07.696269 19838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:07.700377 19838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:07.700467 19838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:07.700610 19838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:26:07.705607 19838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:07.706728 19838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:07.708097 19838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:07.717717 19843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:07.717715 19846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:07.717717 19844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:07.718132 19838 server_base.cc:1034] running on GCE node
I20260809 03:26:07.718504 19838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:07.719125 19838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:07.720331 19838 hybrid_clock.cc:648] HybridClock initialized: now 1786245967720321 us; error 27 us; skew 500 ppm
I20260809 03:26:07.722295 19838 webserver.cc:460] Webserver started at http://127.9.46.193:38565/ using document root <none> and password file <none>
I20260809 03:26:07.722884 19838 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:07.723026 19838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:07.723265 19838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:07.725353 19838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "87232b22b06e44ada0818a346f4bc536"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "4918d8c33f4d20fd1b6427ad4f887cda"
server_key_iv: "e954f1a8df0d7cea4736069ba3e7f810"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.725929 19838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "87232b22b06e44ada0818a346f4bc536"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "4918d8c33f4d20fd1b6427ad4f887cda"
server_key_iv: "e954f1a8df0d7cea4736069ba3e7f810"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.729633 19838 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:26:07.732649 19852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.733868 19838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:26:07.734002 19838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "87232b22b06e44ada0818a346f4bc536"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "4918d8c33f4d20fd1b6427ad4f887cda"
server_key_iv: "e954f1a8df0d7cea4736069ba3e7f810"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.734153 19838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:07.738828 19838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:07.739432 19838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:07.739614 19838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:07.740147 19838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:07.741108 19838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:07.741179 19838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.741250 19838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:07.741302 19838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.750481 19838 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:44367
I20260809 03:26:07.750558 19973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:44367 every 8 connection(s)
I20260809 03:26:07.751447 19838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:26:07.757754 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:07.753368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:44103 (local address 127.9.46.254:33621)
0809 03:26:07.753513 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:07.753516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:07.754641 (+ 1125us) server_negotiation.cc:407] Connection header received
0809 03:26:07.756009 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:07.756012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:07.756050 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:07.756125 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:07.757304 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:07.757308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:07.757310 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:26:07.757321 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:07.757341 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:26:07.757349 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:07.757428 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:07.757607 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 03:26:07.757649 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:26:07.758642 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:07.753754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:33621 (local address 127.9.46.193:44103)
0809 03:26:07.754471 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:07.754514 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:07.755805 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:07.756298 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:07.756304 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:07.756390 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:07.757029 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:26:07.757031 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:07.757076 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:26:07.757146 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:07.757432 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:07.757439 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:07.757444 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 03:26:07.757594 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 03:26:07.757786 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":106,"threads_started":1}
I20260809 03:26:07.759938 19974 heartbeater.cc:347] Connected to a master server at 127.9.46.254:33621
I20260809 03:26:07.760236 19974 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:07.760733 19974 heartbeater.cc:511] Master 127.9.46.254:33621 requested a full tablet report, sending...
I20260809 03:26:07.761647 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 19838
I20260809 03:26:07.761772 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:26:07.762126 9403 external_mini_cluster.cc:1428] Setting key 6332f2e915670ad7314e0d8765a256f0
I20260809 03:26:07.762408 19770 ts_manager.cc:194] Registered new tserver with Master: 87232b22b06e44ada0818a346f4bc536 (127.9.46.193:44367)
I20260809 03:26:07.764285 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33621
--builtin_ntp_servers=127.9.46.212:34555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 03:26:07.877545 19977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:07.877797 19977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:07.877837 19977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:07.881820 19977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:07.881870 19977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:07.881966 19977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:26:07.886626 19977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:07.887676 19977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:07.889007 19977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:07.898103 19985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:07.898164 19982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:07.898134 19983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:07.898231 19977 server_base.cc:1034] running on GCE node
I20260809 03:26:07.898783 19977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:07.899446 19977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:07.900624 19977 hybrid_clock.cc:648] HybridClock initialized: now 1786245967900594 us; error 41 us; skew 500 ppm
I20260809 03:26:07.902508 19977 webserver.cc:460] Webserver started at http://127.9.46.194:42017/ using document root <none> and password file <none>
I20260809 03:26:07.903072 19977 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:07.903174 19977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:07.903398 19977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:07.905501 19977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "bf3e6121b9da458cb99a3376d864992d"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "3ffb8a7eab8f40a9dcca2bab4a7c9e50"
server_key_iv: "f0471f55add0872dcb431c255d32fa9c"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.906062 19977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "bf3e6121b9da458cb99a3376d864992d"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "3ffb8a7eab8f40a9dcca2bab4a7c9e50"
server_key_iv: "f0471f55add0872dcb431c255d32fa9c"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.909668 19977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 03:26:07.912742 19991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.913879 19977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 03:26:07.914013 19977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "bf3e6121b9da458cb99a3376d864992d"
format_stamp: "Formatted at 2026-08-09 03:26:07 on dist-test-slave-65mx"
server_key: "3ffb8a7eab8f40a9dcca2bab4a7c9e50"
server_key_iv: "f0471f55add0872dcb431c255d32fa9c"
server_key_version: "encryptionkey@0"
I20260809 03:26:07.914139 19977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:07.918627 19977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:07.919209 19977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:07.919386 19977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:07.919900 19977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:07.920832 19977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:07.920919 19977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.921005 19977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:07.921056 19977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:07.929875 19977 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:46847
I20260809 03:26:07.929906 20113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:46847 every 8 connection(s)
I20260809 03:26:07.930724 19977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:26:07.930992 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 19977
I20260809 03:26:07.931135 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:26:07.931475 9403 external_mini_cluster.cc:1428] Setting key 15d1a05481a56a83f6e001816056b47a
I20260809 03:26:07.933568 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33621
--builtin_ntp_servers=127.9.46.212:34555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 03:26:07.937278 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:07.932665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:57587 (local address 127.9.46.254:33621)
0809 03:26:07.932809 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:07.932814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:07.934000 (+ 1186us) server_negotiation.cc:407] Connection header received
0809 03:26:07.935314 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:07.935319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:07.935363 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:07.935455 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:07.936753 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:07.936759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:07.936765 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:26:07.936781 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:07.936808 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:26:07.936821 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:07.936920 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:07.937092 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 03:26:07.937144 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:26:07.938135 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:07.933138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:33621 (local address 127.9.46.194:57587)
0809 03:26:07.933842 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:07.933894 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:07.935055 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:07.935682 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:07.935692 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:07.935808 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:07.936404 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:26:07.936408 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:07.936464 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:26:07.936590 (+ 126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:07.936910 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:07.936918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:07.936922 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:26:07.937126 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 03:26:07.937361 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":132,"threads_started":1}
I20260809 03:26:07.939306 20114 heartbeater.cc:347] Connected to a master server at 127.9.46.254:33621
I20260809 03:26:07.939532 20114 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:07.939941 20114 heartbeater.cc:511] Master 127.9.46.254:33621 requested a full tablet report, sending...
I20260809 03:26:07.940999 19769 ts_manager.cc:194] Registered new tserver with Master: bf3e6121b9da458cb99a3376d864992d (127.9.46.194:46847)
W20260809 03:26:08.047287 20117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:08.047577 20117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:08.047628 20117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:08.051753 20117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:08.051805 20117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:08.051860 20117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:26:08.056485 20117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:08.057560 20117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:08.058911 20117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:08.068342 20125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:08.068290 20123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:08.068673 20117 server_base.cc:1034] running on GCE node
W20260809 03:26:08.068280 20122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:08.069523 20117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:08.070317 20117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:08.071609 20117 hybrid_clock.cc:648] HybridClock initialized: now 1786245968071588 us; error 41 us; skew 500 ppm
I20260809 03:26:08.073500 20117 webserver.cc:460] Webserver started at http://127.9.46.195:41793/ using document root <none> and password file <none>
I20260809 03:26:08.074096 20117 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:08.074188 20117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:08.074414 20117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:08.076444 20117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "0fbf516d80474063b3d4719544222b18"
format_stamp: "Formatted at 2026-08-09 03:26:08 on dist-test-slave-65mx"
server_key: "368c2d4826401eb6b861dfd08d6d8d7e"
server_key_iv: "0708ca10907b9ca08915fc3eeca1b256"
server_key_version: "encryptionkey@0"
I20260809 03:26:08.077106 20117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "0fbf516d80474063b3d4719544222b18"
format_stamp: "Formatted at 2026-08-09 03:26:08 on dist-test-slave-65mx"
server_key: "368c2d4826401eb6b861dfd08d6d8d7e"
server_key_iv: "0708ca10907b9ca08915fc3eeca1b256"
server_key_version: "encryptionkey@0"
I20260809 03:26:08.080907 20117 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:26:08.083356 20131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:08.084712 20117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:26:08.084841 20117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "0fbf516d80474063b3d4719544222b18"
format_stamp: "Formatted at 2026-08-09 03:26:08 on dist-test-slave-65mx"
server_key: "368c2d4826401eb6b861dfd08d6d8d7e"
server_key_iv: "0708ca10907b9ca08915fc3eeca1b256"
server_key_version: "encryptionkey@0"
I20260809 03:26:08.084957 20117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:08.089708 20117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:08.090299 20117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:08.090478 20117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:08.090993 20117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:08.091879 20117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:08.091952 20117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:08.092022 20117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:08.092074 20117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:08.100850 20117 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:39203
I20260809 03:26:08.100881 20250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:39203 every 8 connection(s)
I20260809 03:26:08.101722 20117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:26:08.107419 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:08.103449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:56617 (local address 127.9.46.254:33621)
0809 03:26:08.103624 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:08.103629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:08.104662 (+ 1033us) server_negotiation.cc:407] Connection header received
0809 03:26:08.105900 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:08.105902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:08.105940 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:08.106013 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:08.107032 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:08.107035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:08.107037 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:26:08.107049 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:08.107068 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:26:08.107076 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:08.107154 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:08.107273 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 03:26:08.107311 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:26:08.108273 20253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:08.103837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:33621 (local address 127.9.46.195:56617)
0809 03:26:08.104470 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:08.104509 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:08.105724 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:08.106188 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:08.106193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:08.106274 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:08.106787 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:26:08.106789 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:08.106827 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:26:08.106892 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:08.107153 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:08.107160 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:08.107164 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:26:08.107290 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 03:26:08.107493 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":105,"threads_started":1}
I20260809 03:26:08.109408 20251 heartbeater.cc:347] Connected to a master server at 127.9.46.254:33621
I20260809 03:26:08.109632 20251 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:08.110026 20251 heartbeater.cc:511] Master 127.9.46.254:33621 requested a full tablet report, sending...
I20260809 03:26:08.111011 19769 ts_manager.cc:194] Registered new tserver with Master: 0fbf516d80474063b3d4719544222b18 (127.9.46.195:39203)
I20260809 03:26:08.111301 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 20117
I20260809 03:26:08.111426 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:26:08.111698 9403 external_mini_cluster.cc:1428] Setting key 1ca607620c6a349c924bf5faa747a754
I20260809 03:26:08.113699 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 03:26:08.116612 19811 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:26:08.115618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.9.46.254:33621)
0809 03:26:08.115753 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:08.115759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:08.115959 (+ 200us) server_negotiation.cc:407] Connection header received
0809 03:26:08.116122 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:08.116125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:08.116159 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:08.116244 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:08.116465 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42914: BlockingRecv error: recv got EOF from 127.0.0.1:42914 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:26:08.116775 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 19838
I20260809 03:26:08.126133 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 19977
I20260809 03:26:08.133276 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 20117
I20260809 03:26:08.140902 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 19735
2026-08-09T03:26:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (816 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:26:08 dist-test-slave-65mx krb5kdc[20261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:26:08 dist-test-slave-65mx krb5kdc[20261](info): set up 2 sockets
Aug 09 03:26:08 dist-test-slave-65mx krb5kdc[20261](info): commencing operation
krb5kdc: starting...
W20260809 03:26:09.296607 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:26:09 dist-test-slave-65mx krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:26:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:26:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:26:09.428009 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:45395
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:37271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:45395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:26:09.539307 20275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:09.539561 20275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:09.539603 20275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:09.543622 20275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:26:09.543679 20275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:09.543699 20275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:09.543715 20275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:26:09.543731 20275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:26:09.548991 20275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:37271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:45395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:45395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:09.549971 20275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:09.550908 20275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:09.559275 20284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:09.559335 20275 server_base.cc:1034] running on GCE node
W20260809 03:26:09.559356 20281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:09.559298 20280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:09.560019 20275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:09.561182 20275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:09.562363 20275 hybrid_clock.cc:648] HybridClock initialized: now 1786245969562340 us; error 37 us; skew 500 ppm
Aug 09 03:26:09 dist-test-slave-65mx krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:26:09.569690 20275 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:26:09.570633 20275 webserver.cc:460] Webserver started at http://127.9.46.254:39331/ using document root <none> and password file <none>
I20260809 03:26:09.571199 20275 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:09.571288 20275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:09.571524 20275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:09.573637 20275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "e8abdda3353d48159a1d683014e14869"
format_stamp: "Formatted at 2026-08-09 03:26:09 on dist-test-slave-65mx"
server_key: "bff63be46bb6f907cb5c02e012998e02"
server_key_iv: "f6c054fefe0b015a2b5340b983b4fefa"
server_key_version: "encryptionkey@0"
I20260809 03:26:09.574206 20275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "e8abdda3353d48159a1d683014e14869"
format_stamp: "Formatted at 2026-08-09 03:26:09 on dist-test-slave-65mx"
server_key: "bff63be46bb6f907cb5c02e012998e02"
server_key_iv: "f6c054fefe0b015a2b5340b983b4fefa"
server_key_version: "encryptionkey@0"
I20260809 03:26:09.577998 20275 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:26:09.580319 20291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:09.581569 20275 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:26:09.581727 20275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "e8abdda3353d48159a1d683014e14869"
format_stamp: "Formatted at 2026-08-09 03:26:09 on dist-test-slave-65mx"
server_key: "bff63be46bb6f907cb5c02e012998e02"
server_key_iv: "f6c054fefe0b015a2b5340b983b4fefa"
server_key_version: "encryptionkey@0"
I20260809 03:26:09.581840 20275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:09.598928 20275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:09.601778 20275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:09.601989 20275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:09.609509 20275 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:45395
I20260809 03:26:09.609897 20349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:45395 every 8 connection(s)
I20260809 03:26:09.610517 20275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:26:09.613341 20352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:26:09.615191 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 20275
I20260809 03:26:09.615312 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:26:09.615614 9403 external_mini_cluster.cc:1428] Setting key 95dc11ce419cd32de17628ca38b3a428
I20260809 03:26:09.620033 20352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869: Bootstrap starting.
I20260809 03:26:09.622648 20352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869: Neither blocks nor log segments found. Creating new log.
I20260809 03:26:09.623576 20352 log.cc:824] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:26:09 dist-test-slave-65mx krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:26:09.625666 20352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869: No bootstrap required, opened a new log
I20260809 03:26:09.629541 20352 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8abdda3353d48159a1d683014e14869" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45395 } }
I20260809 03:26:09.629768 20352 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:26:09.629830 20352 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8abdda3353d48159a1d683014e14869, State: Initialized, Role: FOLLOWER
I20260809 03:26:09.630190 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:09.617070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48754 (local address 127.9.46.254:45395)
0809 03:26:09.617470 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:09.617504 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:09.617538 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:26:09.618161 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:09.618176 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:09.618484 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:09.618748 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:09.619735 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:09.620791 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:09.622009 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:09.622454 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:09.622567 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:09.625449 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:09.625454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:09.625466 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:26:09.625491 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:09.627211 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:09.627677 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:09.627681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:09.627685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:09.627791 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:09.628088 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:09.628091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:09.628093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:09.628463 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:09.628695 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:09.629004 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 03:26:09.629291 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":130,"threads_started":1}
I20260809 03:26:09.630250 20352 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8abdda3353d48159a1d683014e14869" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45395 } }
I20260809 03:26:09.630415 20352 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:26:09.630481 20352 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:26:09.630548 20352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:26:09.631559 20352 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8abdda3353d48159a1d683014e14869" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45395 } }
I20260809 03:26:09.631896 20352 leader_election.cc:302] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8abdda3353d48159a1d683014e14869; no voters:
I20260809 03:26:09.632228 20352 leader_election.cc:288] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:26:09.632299 20363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:26:09.632501 20363 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [term 1 LEADER]: Becoming Leader. State: Replica: e8abdda3353d48159a1d683014e14869, State: Running, Role: LEADER
I20260809 03:26:09.632813 20363 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8abdda3353d48159a1d683014e14869" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45395 } }
I20260809 03:26:09.633556 20352 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:26:09.636636 20364 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8abdda3353d48159a1d683014e14869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8abdda3353d48159a1d683014e14869" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45395 } } }
I20260809 03:26:09.636767 20364 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [sys.catalog]: This master's current role is: LEADER
I20260809 03:26:09.636996 20365 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8abdda3353d48159a1d683014e14869. Latest consensus state: current_term: 1 leader_uuid: "e8abdda3353d48159a1d683014e14869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8abdda3353d48159a1d683014e14869" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45395 } } }
I20260809 03:26:09.637128 20365 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869 [sys.catalog]: This master's current role is: LEADER
W20260809 03:26:09.638769 20378 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:26:09.638859 20378 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:26:09.638943 20379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:26:09.639598 20379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:26:09.644716 20379 catalog_manager.cc:1347] Generated new cluster ID: 3307b3057ce845f8b3bc990e97215f24
I20260809 03:26:09.644788 20379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:26:09.656823 20379 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:26:09.657747 20379 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:26:09.667183 20379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869: Generated new TSK 0
I20260809 03:26:09.667996 20379 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:26:09.701930 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:45395
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:37271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:26:09.815496 20385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:09.815757 20385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:09.815801 20385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:09.819916 20385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:09.819972 20385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:09.820029 20385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:26:09.825680 20385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:37271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:45395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:09.826691 20385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:09.827651 20385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:09.837772 20392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:09.837837 20391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:09.837822 20394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:09.837919 20385 server_base.cc:1034] running on GCE node
I20260809 03:26:09.838529 20385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:09.839147 20385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:09.840356 20385 hybrid_clock.cc:648] HybridClock initialized: now 1786245969840325 us; error 42 us; skew 500 ppm
Aug 09 03:26:09 dist-test-slave-65mx krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:26:09.843307 20385 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:26:09.844225 20385 webserver.cc:460] Webserver started at http://127.9.46.193:35681/ using document root <none> and password file <none>
I20260809 03:26:09.844848 20385 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:09.844954 20385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:09.845201 20385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:09.847239 20385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "1f0386d958204244ae00054e57d56477"
format_stamp: "Formatted at 2026-08-09 03:26:09 on dist-test-slave-65mx"
server_key: "34d4100dcf8a0f745f571e27a03045cd"
server_key_iv: "446f58ff1eed308b0eb0ade994f71d13"
server_key_version: "encryptionkey@0"
I20260809 03:26:09.847821 20385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "1f0386d958204244ae00054e57d56477"
format_stamp: "Formatted at 2026-08-09 03:26:09 on dist-test-slave-65mx"
server_key: "34d4100dcf8a0f745f571e27a03045cd"
server_key_iv: "446f58ff1eed308b0eb0ade994f71d13"
server_key_version: "encryptionkey@0"
I20260809 03:26:09.852080 20385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:26:09.854689 20401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:09.855957 20385 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:26:09.856055 20385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "1f0386d958204244ae00054e57d56477"
format_stamp: "Formatted at 2026-08-09 03:26:09 on dist-test-slave-65mx"
server_key: "34d4100dcf8a0f745f571e27a03045cd"
server_key_iv: "446f58ff1eed308b0eb0ade994f71d13"
server_key_version: "encryptionkey@0"
I20260809 03:26:09.856197 20385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:09.895145 20385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:09.898649 20385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:09.898864 20385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:09.899494 20385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:09.900333 20385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:09.900408 20385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:09.900481 20385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:09.900537 20385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:09.909883 20385 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:43727
I20260809 03:26:09.909956 20524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:43727 every 8 connection(s)
I20260809 03:26:09.910808 20385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:26:09.919431 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 20385
I20260809 03:26:09.919590 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:26:09.919883 9403 external_mini_cluster.cc:1428] Setting key 1efe3a27e5a0255e757d340d8a1a6fe7
Aug 09 03:26:09 dist-test-slave-65mx krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:26:09.925395 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:09.912669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:46379 (local address 127.9.46.254:45395)
0809 03:26:09.912809 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:09.912813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:09.913673 (+ 860us) server_negotiation.cc:407] Connection header received
0809 03:26:09.915034 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:09.915037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:09.915099 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:09.915207 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:09.916354 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:09.916989 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:09.918121 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:09.918361 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:09.918472 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:09.921372 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:09.921374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:09.921376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:26:09.921397 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:09.923112 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:09.923608 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:09.923614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:09.923618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:09.923721 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:09.924149 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:09.924152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:09.924153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:09.924324 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:09.924401 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:09.925162 (+ 761us) server_negotiation.cc:299] Negotiation successful
0809 03:26:09.925274 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:26:09.926537 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:09.913053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:45395 (local address 127.9.46.193:46379)
0809 03:26:09.913518 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:09.913560 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:09.914826 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:09.915380 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:09.915387 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:09.915739 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:09.916208 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:09.916222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:09.917122 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:09.917133 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:09.917990 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:09.918000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:09.918527 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:09.918531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:09.918697 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:09.919341 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:09.919355 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:09.921172 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:09.923236 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:09.923241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:09.923252 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:09.923504 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:09.923839 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:09.923843 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:09.923847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:09.924050 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:09.924433 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:09.924441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:09.924820 (+ 379us) client_negotiation.cc:769] Sending connection context
0809 03:26:09.925059 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 03:26:09.925428 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":117,"threads_started":1}
I20260809 03:26:09.927983 20525 heartbeater.cc:347] Connected to a master server at 127.9.46.254:45395
I20260809 03:26:09.928246 20525 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:09.928782 20525 heartbeater.cc:511] Master 127.9.46.254:45395 requested a full tablet report, sending...
I20260809 03:26:09.930250 20310 ts_manager.cc:194] Registered new tserver with Master: 1f0386d958204244ae00054e57d56477 (127.9.46.193:43727)
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:26:09.931526 20310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:46379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:26:09.952111 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:45395
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:37271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:26:10.065362 20530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:10.065613 20530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:10.065658 20530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:10.069797 20530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:10.069852 20530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:10.069931 20530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:26:10.074996 20530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:37271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:45395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:10.076103 20530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:10.077447 20530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:10.087540 20539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:10.087540 20536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:10.087560 20537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:10.087901 20530 server_base.cc:1034] running on GCE node
I20260809 03:26:10.088325 20530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:10.089043 20530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:10.090235 20530 hybrid_clock.cc:648] HybridClock initialized: now 1786245970090218 us; error 29 us; skew 500 ppm
Aug 09 03:26:10 dist-test-slave-65mx krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:26:10.093170 20530 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:26:10.094022 20530 webserver.cc:460] Webserver started at http://127.9.46.194:42605/ using document root <none> and password file <none>
I20260809 03:26:10.094530 20530 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:10.094588 20530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:10.094748 20530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:10.096792 20530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162"
format_stamp: "Formatted at 2026-08-09 03:26:10 on dist-test-slave-65mx"
server_key: "67c3c7d4ae9ddba1dd3238868b904131"
server_key_iv: "86934eed4d24a492e672bdd2a05053c9"
server_key_version: "encryptionkey@0"
I20260809 03:26:10.097342 20530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162"
format_stamp: "Formatted at 2026-08-09 03:26:10 on dist-test-slave-65mx"
server_key: "67c3c7d4ae9ddba1dd3238868b904131"
server_key_iv: "86934eed4d24a492e672bdd2a05053c9"
server_key_version: "encryptionkey@0"
I20260809 03:26:10.101163 20530 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:26:10.103564 20548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:10.104936 20530 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:26:10.105041 20530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162"
format_stamp: "Formatted at 2026-08-09 03:26:10 on dist-test-slave-65mx"
server_key: "67c3c7d4ae9ddba1dd3238868b904131"
server_key_iv: "86934eed4d24a492e672bdd2a05053c9"
server_key_version: "encryptionkey@0"
I20260809 03:26:10.105123 20530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:10.122525 20530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:10.125984 20530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:10.126157 20530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:10.126703 20530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:10.127473 20530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:10.127522 20530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:10.127562 20530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:10.127578 20530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:10.137447 20530 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:43869
I20260809 03:26:10.137475 20679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:43869 every 8 connection(s)
I20260809 03:26:10.138296 20530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:26:10.139336 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 20530
I20260809 03:26:10.139462 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:26:10.139776 9403 external_mini_cluster.cc:1428] Setting key 4de9edfe84b7f18bf71812aca1ba6b1b
Aug 09 03:26:10 dist-test-slave-65mx krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
I20260809 03:26:10.154603 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.140152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:36183 (local address 127.9.46.254:45395)
0809 03:26:10.140307 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:10.140312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:10.141327 (+ 1015us) server_negotiation.cc:407] Connection header received
0809 03:26:10.142746 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:10.142750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:10.142802 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:10.142885 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:10.144502 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.145091 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.146923 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.147197 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.147289 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.150266 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:10.150270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:10.150272 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:26:10.150296 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:10.151843 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:10.152375 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:10.152379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:10.152381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:10.152466 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:10.152963 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:10.152966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:10.152968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:10.153169 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:10.153314 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:10.154199 (+ 885us) server_negotiation.cc:299] Negotiation successful
0809 03:26:10.154365 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:26:10.155439 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.140641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:45395 (local address 127.9.46.194:36183)
0809 03:26:10.141162 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:10.141227 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:10.142544 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:10.143070 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:10.143077 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:10.143630 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:10.144333 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.144355 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.145229 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.145236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.146777 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.146792 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.147374 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.147381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.147624 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.148273 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:10.148300 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:10.150064 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:10.151967 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:10.151972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:10.151988 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:10.152263 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:10.152647 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:10.152650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:10.152651 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:10.152836 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:10.153362 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:10.153369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:10.153736 (+ 367us) client_negotiation.cc:769] Sending connection context
0809 03:26:10.154076 (+ 340us) client_negotiation.cc:240] Negotiation successful
0809 03:26:10.154405 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":117,"threads_started":1}
I20260809 03:26:10.156914 20680 heartbeater.cc:347] Connected to a master server at 127.9.46.254:45395
I20260809 03:26:10.157224 20680 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:10.157855 20680 heartbeater.cc:511] Master 127.9.46.254:45395 requested a full tablet report, sending...
I20260809 03:26:10.159181 20310 ts_manager.cc:194] Registered new tserver with Master: 2c1c8b8554ba425b8a1d8c43b3cd2162 (127.9.46.194:43869)
I20260809 03:26:10.159780 20310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:36183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:26:10.172142 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:45395
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:37271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:26:10.284272 20685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:10.284588 20685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:10.284667 20685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:10.288874 20685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:10.288950 20685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:10.289040 20685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:26:10.294512 20685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:37271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:45395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:10.295549 20685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:10.296597 20685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:10.306202 20692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:10.306196 20694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:10.306202 20691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:10.306488 20685 server_base.cc:1034] running on GCE node
I20260809 03:26:10.306998 20685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:10.307704 20685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:10.308936 20685 hybrid_clock.cc:648] HybridClock initialized: now 1786245970308900 us; error 48 us; skew 500 ppm
Aug 09 03:26:10 dist-test-slave-65mx krb5kdc[20261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:26:10.311866 20685 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:26:10.312861 20685 webserver.cc:460] Webserver started at http://127.9.46.195:40361/ using document root <none> and password file <none>
I20260809 03:26:10.313428 20685 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:10.313524 20685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:10.313758 20685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:26:10.315795 20685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "769ea1a647e84f25a14f196b4e27c722"
format_stamp: "Formatted at 2026-08-09 03:26:10 on dist-test-slave-65mx"
server_key: "3d0591799d8494f186f6f6a89216c430"
server_key_iv: "983a43b4aebc040ee0e5b4ebb52ac9e9"
server_key_version: "encryptionkey@0"
I20260809 03:26:10.316387 20685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "769ea1a647e84f25a14f196b4e27c722"
format_stamp: "Formatted at 2026-08-09 03:26:10 on dist-test-slave-65mx"
server_key: "3d0591799d8494f186f6f6a89216c430"
server_key_iv: "983a43b4aebc040ee0e5b4ebb52ac9e9"
server_key_version: "encryptionkey@0"
I20260809 03:26:10.320156 20685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:26:10.322696 20701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:10.323912 20685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:26:10.324054 20685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "769ea1a647e84f25a14f196b4e27c722"
format_stamp: "Formatted at 2026-08-09 03:26:10 on dist-test-slave-65mx"
server_key: "3d0591799d8494f186f6f6a89216c430"
server_key_iv: "983a43b4aebc040ee0e5b4ebb52ac9e9"
server_key_version: "encryptionkey@0"
I20260809 03:26:10.324198 20685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:10.344530 20685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:10.347975 20685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:10.348204 20685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:10.348845 20685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:10.349715 20685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:10.349788 20685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:10.349862 20685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:10.349913 20685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:10.359726 20685 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:40991
I20260809 03:26:10.359874 20823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:40991 every 8 connection(s)
I20260809 03:26:10.360746 20685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:26:10.369424 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 20685
I20260809 03:26:10.369582 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:26:10.369982 9403 external_mini_cluster.cc:1428] Setting key 172fbb53b7aebedbacdcdc82b83cee1a
Aug 09 03:26:10 dist-test-slave-65mx krb5kdc[20261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:26:10.375859 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.362569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:35705 (local address 127.9.46.254:45395)
0809 03:26:10.362720 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:10.362724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:10.363620 (+ 896us) server_negotiation.cc:407] Connection header received
0809 03:26:10.365242 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:10.365245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:10.365310 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:10.365401 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:10.366636 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.367238 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.368500 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.368828 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.368959 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.372154 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:10.372159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:10.372162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:26:10.372193 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:10.373930 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:10.374454 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:10.374456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:10.374458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:10.374540 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:10.374855 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:10.374857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:10.374858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:10.375089 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:10.375171 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:10.375627 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 03:26:10.375731 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 03:26:10.376742 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.362953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:45395 (local address 127.9.46.195:35705)
0809 03:26:10.363448 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:10.363501 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:10.365024 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:10.365577 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:10.365584 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:10.365956 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:10.366466 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.366491 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.367383 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.367393 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.368360 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.368370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.369006 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.369011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.369273 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.369972 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:10.369999 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:10.371989 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:10.374065 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:10.374070 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:10.374084 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:10.374358 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:10.374630 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:10.374632 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:10.374633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:10.374773 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:10.375180 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:10.375185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:10.375421 (+ 236us) client_negotiation.cc:769] Sending connection context
0809 03:26:10.375577 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 03:26:10.375847 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":115,"threads_started":1}
I20260809 03:26:10.377966 20824 heartbeater.cc:347] Connected to a master server at 127.9.46.254:45395
I20260809 03:26:10.378237 20824 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:10.378747 20824 heartbeater.cc:511] Master 127.9.46.254:45395 requested a full tablet report, sending...
I20260809 03:26:10.379740 20310 ts_manager.cc:194] Registered new tserver with Master: 769ea1a647e84f25a14f196b4e27c722 (127.9.46.195:40991)
I20260809 03:26:10.380306 20310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:35705
I20260809 03:26:10.384371 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:26:10.394984 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.386539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.9.46.254:45395)
0809 03:26:10.386665 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:10.386669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:10.386942 (+ 273us) server_negotiation.cc:407] Connection header received
0809 03:26:10.387132 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:10.387135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:10.387179 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:10.387276 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:10.388068 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.388699 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.389648 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.389886 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.390006 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.391394 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:10.391400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:10.391409 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:26:10.391433 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:10.393013 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:10.393485 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:10.393492 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:10.393497 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:10.393628 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:10.393943 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:10.393948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:10.393950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:10.394191 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:10.394353 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:10.394625 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 03:26:10.394751 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:26:10.398254 20310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:26:10.401464 20310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:26:10.415777 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.409802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:43727 (local address 127.0.0.1:59806)
0809 03:26:10.410147 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:10.410201 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:10.410475 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:10.411092 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:10.411100 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:10.411126 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:26:10.411279 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.411289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.412861 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.412864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.414327 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.414337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.415242 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.415246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.415352 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.415389 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 03:26:10.415502 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 03:26:10.415625 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":113,"thread_start_us":94,"threads_started":1}
I20260809 03:26:10.416005 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.409995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:43869 (local address 127.0.0.1:34212)
0809 03:26:10.410680 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:10.410695 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:10.410782 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:10.411422 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:10.411425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:10.411444 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:26:10.411566 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.411574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.413285 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.413288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.414552 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.414561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.415620 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.415624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.415707 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.415730 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:26:10.415780 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 03:26:10.415860 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":127,"thread_start_us":136,"threads_started":1}
I20260809 03:26:10.415823 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.409954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.9.46.193:43727)
0809 03:26:10.410539 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:10.410557 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:10.410585 (+ 28us) server_negotiation.cc:407] Connection header received
0809 03:26:10.410669 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:10.410674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:10.410894 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:10.411453 (+ 559us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:26:10.411661 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.412727 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.414465 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.415106 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.415229 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.415381 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:10.415542 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 03:26:10.415662 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":63,"threads_started":1}
I20260809 03:26:10.416762 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.410579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.9.46.194:43869)
0809 03:26:10.410905 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:10.410923 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:10.410947 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:26:10.411025 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:10.411030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:10.411230 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:10.411809 (+ 579us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:26:10.411977 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.413125 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.414674 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.415486 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.416296 (+ 810us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.416456 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:10.416508 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 03:26:10.416649 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":129,"threads_started":1}
I20260809 03:26:10.416999 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.411035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:40991 (local address 127.0.0.1:32940)
0809 03:26:10.411966 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:10.411981 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:10.412080 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:10.412500 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:10.412503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:10.412529 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:26:10.412636 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.412641 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.413919 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.413923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.415306 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.415318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.416611 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.416614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.416697 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.416719 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:26:10.416798 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 03:26:10.416863 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":100,"thread_start_us":101,"threads_started":1}
I20260809 03:26:10.417654 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.411420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32940 (local address 127.9.46.195:40991)
0809 03:26:10.411830 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:10.411841 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:10.412093 (+ 252us) server_negotiation.cc:407] Connection header received
0809 03:26:10.412173 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:10.412178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:10.412350 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:10.412499 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:26:10.412746 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.413634 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.415462 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.416273 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.416375 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.416523 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:10.417394 (+ 871us) server_negotiation.cc:299] Negotiation successful
0809 03:26:10.417500 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":102,"threads_started":1}
I20260809 03:26:10.419318 20603 tablet_service.cc:1467] Processing CreateTablet for tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf (DEFAULT_TABLE table=test-table [id=1ba9a02bab3a4d08b77a1d20382c240f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:26:10.419358 20451 tablet_service.cc:1467] Processing CreateTablet for tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf (DEFAULT_TABLE table=test-table [id=1ba9a02bab3a4d08b77a1d20382c240f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:26:10.420305 20603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:26:10.420408 20451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:26:10.420189 20754 tablet_service.cc:1467] Processing CreateTablet for tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf (DEFAULT_TABLE table=test-table [id=1ba9a02bab3a4d08b77a1d20382c240f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:26:10.421236 20754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:26:10.426558 20842 tablet_bootstrap.cc:492] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: Bootstrap starting.
I20260809 03:26:10.427997 20843 tablet_bootstrap.cc:492] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: Bootstrap starting.
I20260809 03:26:10.428951 20844 tablet_bootstrap.cc:492] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477: Bootstrap starting.
I20260809 03:26:10.429761 20842 tablet_bootstrap.cc:654] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: Neither blocks nor log segments found. Creating new log.
I20260809 03:26:10.430184 20843 tablet_bootstrap.cc:654] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: Neither blocks nor log segments found. Creating new log.
I20260809 03:26:10.430729 20842 log.cc:824] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: Log is configured to *not* fsync() on all Append() calls
I20260809 03:26:10.430951 20844 tablet_bootstrap.cc:654] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477: Neither blocks nor log segments found. Creating new log.
I20260809 03:26:10.431384 20843 log.cc:824] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: Log is configured to *not* fsync() on all Append() calls
I20260809 03:26:10.431819 20844 log.cc:824] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477: Log is configured to *not* fsync() on all Append() calls
I20260809 03:26:10.433007 20842 tablet_bootstrap.cc:492] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: No bootstrap required, opened a new log
I20260809 03:26:10.433214 20842 ts_tablet_manager.cc:1397] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:26:10.433472 20843 tablet_bootstrap.cc:492] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: No bootstrap required, opened a new log
I20260809 03:26:10.433648 20843 ts_tablet_manager.cc:1397] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:26:10.434795 20844 tablet_bootstrap.cc:492] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477: No bootstrap required, opened a new log
I20260809 03:26:10.434991 20844 ts_tablet_manager.cc:1397] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:26:10.437315 20842 raft_consensus.cc:348] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 } } peers { permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 } } peers { permanent_uuid: "769ea1a647e84f25a14f196b4e27c722" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 40991 } }
I20260809 03:26:10.437327 20843 raft_consensus.cc:348] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 } } peers { permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 } } peers { permanent_uuid: "769ea1a647e84f25a14f196b4e27c722" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 40991 } }
I20260809 03:26:10.437598 20842 raft_consensus.cc:374] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:26:10.437595 20843 raft_consensus.cc:374] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:26:10.437656 20843 raft_consensus.cc:729] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769ea1a647e84f25a14f196b4e27c722, State: Initialized, Role: FOLLOWER
I20260809 03:26:10.437657 20842 raft_consensus.cc:729] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c1c8b8554ba425b8a1d8c43b3cd2162, State: Initialized, Role: FOLLOWER
I20260809 03:26:10.438114 20843 consensus_queue.cc:260] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 } } peers { permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 } } peers { permanent_uuid: "769ea1a647e84f25a14f196b4e27c722" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 40991 } }
I20260809 03:26:10.438114 20842 consensus_queue.cc:260] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 } } peers { permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 } } peers { permanent_uuid: "769ea1a647e84f25a14f196b4e27c722" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 40991 } }
I20260809 03:26:10.438822 20680 heartbeater.cc:503] Master 127.9.46.254:45395 was elected leader, sending a full tablet report...
I20260809 03:26:10.438926 20844 raft_consensus.cc:348] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 } } peers { permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 } } peers { permanent_uuid: "769ea1a647e84f25a14f196b4e27c722" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 40991 } }
I20260809 03:26:10.439108 20824 heartbeater.cc:503] Master 127.9.46.254:45395 was elected leader, sending a full tablet report...
I20260809 03:26:10.439133 20843 ts_tablet_manager.cc:1428] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:26:10.439177 20844 raft_consensus.cc:374] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:26:10.439224 20844 raft_consensus.cc:729] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0386d958204244ae00054e57d56477, State: Initialized, Role: FOLLOWER
I20260809 03:26:10.439627 20844 consensus_queue.cc:260] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 } } peers { permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 } } peers { permanent_uuid: "769ea1a647e84f25a14f196b4e27c722" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 40991 } }
I20260809 03:26:10.440143 20842 ts_tablet_manager.cc:1428] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:26:10.440582 20844 ts_tablet_manager.cc:1428] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:26:10.440601 20525 heartbeater.cc:503] Master 127.9.46.254:45395 was elected leader, sending a full tablet report...
W20260809 03:26:10.611682 20825 tablet.cc:2369] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:26:10.639441 20681 tablet.cc:2369] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:26:10.662202 20526 tablet.cc:2369] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:26:10.761164 20849 raft_consensus.cc:482] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:26:10.761369 20849 raft_consensus.cc:504] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 } } peers { permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 } } peers { permanent_uuid: "769ea1a647e84f25a14f196b4e27c722" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 40991 } }
I20260809 03:26:10.762485 20849 leader_election.cc:288] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f0386d958204244ae00054e57d56477 (127.9.46.193:43727), 2c1c8b8554ba425b8a1d8c43b3cd2162 (127.9.46.194:43869)
I20260809 03:26:10.766621 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.762801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:43869 (local address 127.9.46.195:58445)
0809 03:26:10.762943 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:10.762961 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:10.763091 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:10.763484 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:10.763490 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:10.763519 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:26:10.763661 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.763668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.764494 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.764497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.765429 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.765439 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.766187 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.766190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.766300 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.766326 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 03:26:10.766410 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:26:10.766481 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 03:26:10.766659 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.762937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:58445 (local address 127.9.46.194:43869)
0809 03:26:10.763169 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:10.763176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:10.763193 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:26:10.763253 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:10.763256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:10.763304 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:10.763403 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:26:10.763790 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.764384 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.765565 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.766066 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.766162 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.766210 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:10.766446 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 03:26:10.766522 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260809 03:26:10.766928 20851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.762801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:43727 (local address 127.9.46.195:34679)
0809 03:26:10.763379 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:10.763394 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:10.763483 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:10.763779 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:10.763782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:10.763799 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:26:10.763918 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.763924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.764854 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.764857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.765717 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:10.765726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.766458 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.766462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:10.766580 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.766603 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:26:10.766685 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:26:10.766757 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":144,"thread_start_us":183,"threads_started":1}
I20260809 03:26:10.766956 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.762932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:34679 (local address 127.9.46.193:43727)
0809 03:26:10.763225 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:10.763230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:10.763501 (+ 271us) server_negotiation.cc:407] Connection header received
0809 03:26:10.763595 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:10.763597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:10.763653 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:10.763739 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:26:10.764044 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.764745 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.765844 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.766330 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.766446 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.766493 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:10.766724 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 03:26:10.766814 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260809 03:26:10.767470 20626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f1f8e3ce33540f69e0b9a4aedee5bdf" candidate_uuid: "769ea1a647e84f25a14f196b4e27c722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" is_pre_election: true
I20260809 03:26:10.767592 20473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f1f8e3ce33540f69e0b9a4aedee5bdf" candidate_uuid: "769ea1a647e84f25a14f196b4e27c722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0386d958204244ae00054e57d56477" is_pre_election: true
I20260809 03:26:10.767838 20626 raft_consensus.cc:2454] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 769ea1a647e84f25a14f196b4e27c722 in term 0.
I20260809 03:26:10.767836 20473 raft_consensus.cc:2454] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 769ea1a647e84f25a14f196b4e27c722 in term 0.
I20260809 03:26:10.768390 20704 leader_election.cc:302] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f0386d958204244ae00054e57d56477, 2c1c8b8554ba425b8a1d8c43b3cd2162, 769ea1a647e84f25a14f196b4e27c722; no voters:
I20260809 03:26:10.768745 20849 raft_consensus.cc:2781] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:26:10.768848 20849 raft_consensus.cc:482] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:26:10.768911 20849 raft_consensus.cc:3037] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:26:10.770016 20849 raft_consensus.cc:504] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 } } peers { permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 } } peers { permanent_uuid: "769ea1a647e84f25a14f196b4e27c722" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 40991 } }
I20260809 03:26:10.770385 20849 leader_election.cc:288] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [CANDIDATE]: Term 1 election: Requested vote from peers 1f0386d958204244ae00054e57d56477 (127.9.46.193:43727), 2c1c8b8554ba425b8a1d8c43b3cd2162 (127.9.46.194:43869)
I20260809 03:26:10.770795 20626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f1f8e3ce33540f69e0b9a4aedee5bdf" candidate_uuid: "769ea1a647e84f25a14f196b4e27c722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162"
I20260809 03:26:10.770843 20473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f1f8e3ce33540f69e0b9a4aedee5bdf" candidate_uuid: "769ea1a647e84f25a14f196b4e27c722" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0386d958204244ae00054e57d56477"
I20260809 03:26:10.770962 20626 raft_consensus.cc:3037] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:26:10.770980 20473 raft_consensus.cc:3037] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:26:10.772049 20473 raft_consensus.cc:2454] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 769ea1a647e84f25a14f196b4e27c722 in term 1.
I20260809 03:26:10.772070 20626 raft_consensus.cc:2454] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 769ea1a647e84f25a14f196b4e27c722 in term 1.
I20260809 03:26:10.772385 20705 leader_election.cc:302] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c1c8b8554ba425b8a1d8c43b3cd2162, 769ea1a647e84f25a14f196b4e27c722; no voters:
I20260809 03:26:10.772570 20849 raft_consensus.cc:2781] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:26:10.772789 20849 raft_consensus.cc:686] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 1 LEADER]: Becoming Leader. State: Replica: 769ea1a647e84f25a14f196b4e27c722, State: Running, Role: LEADER
I20260809 03:26:10.773118 20849 consensus_queue.cc:237] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 } } peers { permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 } } peers { permanent_uuid: "769ea1a647e84f25a14f196b4e27c722" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 40991 } }
I20260809 03:26:10.776476 20310 catalog_manager.cc:5495] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 reported cstate change: term changed from 0 to 1, leader changed from <none> to 769ea1a647e84f25a14f196b4e27c722 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "769ea1a647e84f25a14f196b4e27c722" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "769ea1a647e84f25a14f196b4e27c722" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 40991 } health_report { overall_health: HEALTHY } } }
I20260809 03:26:10.871639 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.868079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32942 (local address 127.9.46.195:40991)
0809 03:26:10.868220 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:10.868224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:10.868252 (+ 28us) server_negotiation.cc:407] Connection header received
0809 03:26:10.868419 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:10.868422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:10.868483 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:10.868614 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:26:10.868833 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.869343 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.870248 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:10.870453 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:10.870528 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:10.870831 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:26:10.871269 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:26:10.871361 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:10.871453 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 03:26:10.871507 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:26:10.877342 20473 raft_consensus.cc:1261] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477 [term 1 FOLLOWER]: Refusing update from remote peer 769ea1a647e84f25a14f196b4e27c722: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:26:10.877360 20626 raft_consensus.cc:1261] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162 [term 1 FOLLOWER]: Refusing update from remote peer 769ea1a647e84f25a14f196b4e27c722: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:26:10.878115 20849 consensus_queue.cc:1035] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0386d958204244ae00054e57d56477" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:26:10.878319 20852 consensus_queue.cc:1035] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c1c8b8554ba425b8a1d8c43b3cd2162" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 43869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:26:10.887627 20857 mvcc.cc:205] Tried to move back new op lower bound from 7316463496706314240 to 7316463496289378304. Current Snapshot: MvccSnapshot[applied={T|T < 7316463496706314240}]
I20260809 03:26:10.894486 20310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:26:10.894704 20310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:26:10.897449 20310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8abdda3353d48159a1d683014e14869: Sending DeleteTablet for 3 replicas of tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf
I20260809 03:26:10.898365 20451 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:26:10 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:59806
I20260809 03:26:10.898371 20603 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:26:10 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:34212
I20260809 03:26:10.898887 20867 tablet_replica.cc:330] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477: stopping tablet replica
I20260809 03:26:10.898895 20868 tablet_replica.cc:330] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: stopping tablet replica
I20260809 03:26:10.898922 20754 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:26:10 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:32940
I20260809 03:26:10.899219 20868 raft_consensus.cc:2229] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:26:10.899384 20867 raft_consensus.cc:2229] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:26:10.899555 20868 raft_consensus.cc:2258] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:26:10.899897 20867 raft_consensus.cc:2258] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 1f0386d958204244ae00054e57d56477 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:26:10.900177 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 20385
I20260809 03:26:10.901463 20868 ts_tablet_manager.cc:1905] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:26:10.904402 20868 ts_tablet_manager.cc:1918] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:26:10.904508 20868 log.cc:1196] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/7f1f8e3ce33540f69e0b9a4aedee5bdf
I20260809 03:26:10.904884 20868 ts_tablet_manager.cc:1939] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 2c1c8b8554ba425b8a1d8c43b3cd2162: Deleting consensus metadata
I20260809 03:26:10.906173 20295 catalog_manager.cc:4841] TS 2c1c8b8554ba425b8a1d8c43b3cd2162 (127.9.46.194:43869): tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf (table test-table [id=1ba9a02bab3a4d08b77a1d20382c240f]) successfully deleted
I20260809 03:26:10.908453 20869 tablet_replica.cc:330] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: stopping tablet replica
I20260809 03:26:10.908850 20869 raft_consensus.cc:2229] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:26:10.909246 20869 raft_consensus.cc:2258] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:26:10.910188 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 20530
W20260809 03:26:10.910210 20294 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:43727 (error 108)
I20260809 03:26:10.912177 20869 ts_tablet_manager.cc:1905] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:26:10.915609 20869 ts_tablet_manager.cc:1918] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:26:10.915725 20869 log.cc:1196] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/7f1f8e3ce33540f69e0b9a4aedee5bdf
I20260809 03:26:10.916078 20869 ts_tablet_manager.cc:1939] T 7f1f8e3ce33540f69e0b9a4aedee5bdf P 769ea1a647e84f25a14f196b4e27c722: Deleting consensus metadata
I20260809 03:26:10.916715 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:10.916303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:43727 (local address 127.0.0.1:59816)
0809 03:26:10.916511 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:10.916628 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:43727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260809 03:26:10.916980 20294 catalog_manager.cc:4584] TS 1f0386d958204244ae00054e57d56477 (127.9.46.193:43727): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf: Network error: Client connection negotiation failed: client connection to 127.9.46.193:43727: connect: Connection refused (error 111)
I20260809 03:26:10.917613 20295 catalog_manager.cc:4841] TS 769ea1a647e84f25a14f196b4e27c722 (127.9.46.195:40991): tablet 7f1f8e3ce33540f69e0b9a4aedee5bdf (table test-table [id=1ba9a02bab3a4d08b77a1d20382c240f]) successfully deleted
I20260809 03:26:10.919272 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 20685
I20260809 03:26:10.927635 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 20275
2026-08-09T03:26:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2789 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:26:10 dist-test-slave-65mx krb5kdc[20873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:26:10 dist-test-slave-65mx krb5kdc[20873](info): set up 2 sockets
Aug 09 03:26:10 dist-test-slave-65mx krb5kdc[20873](info): commencing operation
krb5kdc: starting...
W20260809 03:26:12.079304 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:26:12 dist-test-slave-65mx krb5kdc[20873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786245972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:26:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:26:12Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.46.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.46.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/rangeradmin_127.9.46.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/rangeradmin_127.9.46.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.46.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.46.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/rangerlookup_127.9.46.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/rangerlookup_127.9.46.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.46.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.46.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/HTTP_127.9.46.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/HTTP_127.9.46.212@KRBTEST.COM.keytab.
I20260809 03:26:12.262904 9403 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOU4Lix/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/postgres -l logfile start
2026-08-09 03:26:14.864 UTC [20904] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 03:26:14.864 UTC [20904] LOG: listening on IPv4 address "127.9.46.212", port 51995
2026-08-09 03:26:14.868 UTC [20904] LOG: listening on Unix socket "/tmp/.s.PGSQL.51995"
2026-08-09 03:26:14.905 UTC [20909] LOG: database system was shut down at 2026-08-09 03:26:13 UTC
2026-08-09 03:26:14.909 UTC [20904] LOG: database system is ready to accept connections
I20260809 03:26:15.906180 9403 mini_postgres.cc:96] Postgres bound to 51995
2026-08-09 03:26:15.914 UTC [20917] FATAL: database "slave" does not exist
127.9.46.212:51995 - accepting connections
I20260809 03:26:15.915387 9403 mini_ranger.cc:161] Starting Ranger...
I20260809 03:26:15.931384 9403 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 03:26:16.413233 9403 mini_ranger.cc:87] Created ranger Postgres database
I20260809 03:26:16.413332 9403 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-admin
2026-08-09 03:26:16,790 [I] DB FLAVOR :POSTGRES
2026-08-09 03:26:16,791 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:26:16,791 [I] Checking connection..
2026-08-09 03:26:16,791 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 03:26:17,073 [I] Checking connection passed.
2026-08-09 03:26:17,073 [I] --------- Verifying version history table ---------
2026-08-09 03:26:17,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:26:17,350 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:26:17,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:26:17,634 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:26:17,634 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 03:26:17,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 03:26:17.890 UTC [20985] WARNING: there is no transaction in progress
2026-08-09 03:26:17,898 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 03:26:17,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:26:18,145 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 03:26:18,145 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:26:18,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 03:26:18,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 03:26:18,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-65mx.c.gcp-upstream.internal','N') ;"
2026-08-09 03:26:18,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:26:19,169 [I] Table x_portal_user does not exist in database ranger
2026-08-09 03:26:19,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:26:19,449 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 03:26:19,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:26:19,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:26:19,949 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 03:26:19,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 03:26:20.922 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:20.939 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:20.957 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:20.975 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:20.995 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.016 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.076 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.083 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.099 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.109 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.120 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.134 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.144 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.150 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.157 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21.164 UTC [21107] WARNING: there is no transaction in progress
2026-08-09 03:26:21,811 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 03:26:21,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 03:26:22,061 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 03:26:22,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:26:22,319 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 03:26:22,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:26:22,600 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 03:26:22,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:26:22,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:26:23,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-65mx.c.gcp-upstream.internal';"
2026-08-09 03:26:23,331 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 03:26:23,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 03:26:23,584 [I] DB_PATCHES have already been applied
I20260809 03:26:23.590567 9403 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOU4Lix/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 03:26:23.590663 9403 mini_ranger.cc:193] Using host: 127.9.46.212
I20260809 03:26:23.595121 9403 mini_ranger.cc:227] Ranger admin URL: http://127.9.46.212:36361
Aug 09, 2026 3:26:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:26:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:26:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:26:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:26:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:26:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:26:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 03:26:24.751950 9403 mini_ranger.cc:160] Time spent starting Ranger: real 8.837s user 0.000s sys 0.013s
Aug 09, 2026 3:26:44 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:26:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 3:26:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:26:44 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:26:45 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 03:26:46.325443 9403 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.46.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.46.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/rangerkms_127.9.46.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/rangerkms_127.9.46.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.46.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.46.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.46.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/HTTP_127.9.46.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.46.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/HTTP_127.9.46.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 03:26:46 dist-test-slave-65mx krb5kdc[20873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246006, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 03:26:46.414752 9403 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 03:26:46.425199 9403 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 03:26:47.315464 9403 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 03:26:47.315567 9403 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-kms
I20260809 03:26:47.315672 9403 mini_ranger_kms.cc:227] Using postgres at 127.9.46.212:51995
2026-08-09 03:26:48,002 [I] DB FLAVOR :POSTGRES
2026-08-09 03:26:48,002 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:26:48,002 [I] Checking connection
2026-08-09 03:26:48,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 03:26:48,266 [I] connection success
2026-08-09 03:26:48,267 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 03:26:48,267 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 03:26:48,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 03:26:48,506 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 03:26:48,507 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:26:48,507 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 03:26:48,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:51995/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 03:26:48,768 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 03:26:49.281145 9403 mini_ranger_kms.cc:310] Created kms service
I20260809 03:26:49.463176 9403 mini_ranger_kms.cc:326] Created kudu user
I20260809 03:26:49.508121 9403 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 03:26:49 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246006, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:26:49.939704 9403 mini_ranger_kms.cc:384] Added ranger policy
I20260809 03:26:49.939889 9403 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 03:26:49.939925 9403 mini_ranger_kms.cc:240] Using host: 127.9.46.212
I20260809 03:26:49.944967 9403 mini_ranger_kms.cc:276] Ranger KMS PID: 21307
I20260809 03:26:49.945132 9403 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.46.212:54567
Aug 09, 2026 3:26:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:26:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:26:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:26:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:26:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:26:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 03:26:52.314340 9403 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.900s user 0.013s sys 0.343s
I20260809 03:26:52.314460 9403 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 03:26:52.314594 9403 mini_ranger_kms.cc:401] 127.9.46.212:54567/kms/v1/keys
Aug 09 03:26:52 dist-test-slave-65mx krb5kdc[20873](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246012, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.46.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 03:26:52 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246012, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.46.212@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
Aug 09 03:26:53 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246012, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.46.212@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
Aug 09 03:26:55 dist-test-slave-65mx krb5kdc[20873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:26:55.763522 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46197
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:46197
--ranger_config_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:54567/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:26:55.882863 21365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:55.883143 21365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:55.883184 21365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:55.887235 21365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:26:55.887303 21365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:55.887323 21365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:55.887339 21365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:26:55.887355 21365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:26:55.893085 21365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.46.212:54567/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:46197
--ranger_config_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:55.894120 21365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:55.895079 21365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:26:55.904261 21365 server_base.cc:1034] running on GCE node
W20260809 03:26:55.904163 21372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:55.904193 21374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:55.904193 21371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:55.905064 21365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:55.906243 21365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:55.907440 21365 hybrid_clock.cc:648] HybridClock initialized: now 1786246015907427 us; error 38 us; skew 500 ppm
Aug 09 03:26:55 dist-test-slave-65mx krb5kdc[20873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246015, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:26:55.910322 21365 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:26:55.911237 21365 webserver.cc:460] Webserver started at http://127.9.46.254:46637/ using document root <none> and password file <none>
I20260809 03:26:55.911803 21365 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:55.911888 21365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:55.912124 21365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:26:55 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246015, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:26:55.963889 21365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "3d314d05d3ec4a91ba66a60504fe7d33"
format_stamp: "Formatted at 2026-08-09 03:26:55 on dist-test-slave-65mx"
server_key: "9b24d968cce06204cce2705a9acc18ac"
server_key_iv: "402a5c9cc84188504b14ad4c39685eed"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:55.964730 21365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "3d314d05d3ec4a91ba66a60504fe7d33"
format_stamp: "Formatted at 2026-08-09 03:26:55 on dist-test-slave-65mx"
server_key: "9b24d968cce06204cce2705a9acc18ac"
server_key_iv: "402a5c9cc84188504b14ad4c39685eed"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:55.970252 21365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 03:26:55.989130 21381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:55.990918 21365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:26:55.991072 21365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "3d314d05d3ec4a91ba66a60504fe7d33"
format_stamp: "Formatted at 2026-08-09 03:26:55 on dist-test-slave-65mx"
server_key: "9b24d968cce06204cce2705a9acc18ac"
server_key_iv: "402a5c9cc84188504b14ad4c39685eed"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:55.991228 21365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:56.016825 21365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:56.020298 21365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:56.020604 21365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:56.030685 21365 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:46197
I20260809 03:26:56.030711 21439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:46197 every 8 connection(s)
I20260809 03:26:56.031993 21365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:26:56.032326 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 21365
I20260809 03:26:56.032475 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:26:56.032841 9403 external_mini_cluster.cc:1428] Setting key b10ef342e6ca482ee6c85a70b0e63286
I20260809 03:26:56.035424 21440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:26:56.042382 21440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33: Bootstrap starting.
Aug 09 03:26:56 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:26:56.045265 21440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33: Neither blocks nor log segments found. Creating new log.
I20260809 03:26:56.046712 21440 log.cc:824] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33: Log is configured to *not* fsync() on all Append() calls
I20260809 03:26:56.049120 21440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33: No bootstrap required, opened a new log
I20260809 03:26:56.050379 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:56.034781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52788 (local address 127.9.46.254:46197)
0809 03:26:56.035308 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:56.035327 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:56.035371 (+ 44us) server_negotiation.cc:407] Connection header received
0809 03:26:56.036241 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:56.036263 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:56.036722 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:56.037105 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:56.038012 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:56.039595 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:56.040942 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:56.041653 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:56.041825 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:56.045005 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:56.045013 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:56.045031 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:26:56.045063 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:56.047372 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:56.047860 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:56.047870 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:56.047875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:56.047978 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:56.048258 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:56.048264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:56.048265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:56.048724 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:56.048935 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:56.049285 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 03:26:56.049584 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":198,"threads_started":1}
I20260809 03:26:56.053452 21440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d314d05d3ec4a91ba66a60504fe7d33" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46197 } }
I20260809 03:26:56.053772 21440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:26:56.053859 21440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d314d05d3ec4a91ba66a60504fe7d33, State: Initialized, Role: FOLLOWER
I20260809 03:26:56.054322 21440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d314d05d3ec4a91ba66a60504fe7d33" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46197 } }
I20260809 03:26:56.054481 21440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:26:56.054548 21440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:26:56.054641 21440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:26:56.055779 21440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d314d05d3ec4a91ba66a60504fe7d33" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46197 } }
I20260809 03:26:56.056145 21440 leader_election.cc:302] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d314d05d3ec4a91ba66a60504fe7d33; no voters:
I20260809 03:26:56.056499 21440 leader_election.cc:288] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:26:56.056702 21445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:26:56.057041 21445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [term 1 LEADER]: Becoming Leader. State: Replica: 3d314d05d3ec4a91ba66a60504fe7d33, State: Running, Role: LEADER
I20260809 03:26:56.057417 21445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d314d05d3ec4a91ba66a60504fe7d33" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46197 } }
I20260809 03:26:56.057868 21440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:26:56.058586 21445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d314d05d3ec4a91ba66a60504fe7d33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d314d05d3ec4a91ba66a60504fe7d33" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46197 } } }
I20260809 03:26:56.058794 21445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [sys.catalog]: This master's current role is: LEADER
I20260809 03:26:56.059149 21445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d314d05d3ec4a91ba66a60504fe7d33. Latest consensus state: current_term: 1 leader_uuid: "3d314d05d3ec4a91ba66a60504fe7d33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d314d05d3ec4a91ba66a60504fe7d33" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46197 } } }
I20260809 03:26:56.059304 21445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33 [sys.catalog]: This master's current role is: LEADER
I20260809 03:26:56.059358 21451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:26:56.062704 21451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:26:56.063309 21440 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 03:26:56.073396 21451 catalog_manager.cc:1347] Generated new cluster ID: cce4343f9dec4ec39b1167b53dc3d24e
I20260809 03:26:56.073545 21451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:26:56.096349 21451 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:26:56.097554 21451 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:26:56.105450 21451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33: Generated new TSK 0
I20260809 03:26:56.106267 21451 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:26:56.872269 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:54567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46197
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:26:57.003047 21485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:57.003346 21485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:57.003438 21485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:57.007658 21485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:57.007747 21485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:57.007853 21485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:26:57.013208 21485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.46.212:54567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:57.014324 21485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:57.015388 21485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:57.024924 21494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:57.024947 21491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:57.025424 21492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:57.026369 21485 server_base.cc:1034] running on GCE node
I20260809 03:26:57.026870 21485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:57.027733 21485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:57.028952 21485 hybrid_clock.cc:648] HybridClock initialized: now 1786246017028914 us; error 48 us; skew 500 ppm
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:26:57.032765 21485 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:26:57.033797 21485 webserver.cc:460] Webserver started at http://127.9.46.193:34851/ using document root <none> and password file <none>
I20260809 03:26:57.034457 21485 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:57.034540 21485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:57.034770 21485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:26:57.053526 21485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "83ecdf4b1df34748a16614b644b2e94d"
format_stamp: "Formatted at 2026-08-09 03:26:57 on dist-test-slave-65mx"
server_key: "6e775ff69e75f614a44a32cf2fc14f28"
server_key_iv: "6b2121f83d44e50e0d1863987ee81800"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:57.054208 21485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "83ecdf4b1df34748a16614b644b2e94d"
format_stamp: "Formatted at 2026-08-09 03:26:57 on dist-test-slave-65mx"
server_key: "6e775ff69e75f614a44a32cf2fc14f28"
server_key_iv: "6b2121f83d44e50e0d1863987ee81800"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:57.058907 21485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:26:57.076961 21502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:57.078845 21485 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:26:57.078989 21485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "83ecdf4b1df34748a16614b644b2e94d"
format_stamp: "Formatted at 2026-08-09 03:26:57 on dist-test-slave-65mx"
server_key: "6e775ff69e75f614a44a32cf2fc14f28"
server_key_iv: "6b2121f83d44e50e0d1863987ee81800"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:57.079137 21485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:57.102891 21485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:57.107167 21485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:57.107398 21485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:57.108203 21485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:57.109431 21485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:57.109530 21485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:57.109690 21485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:57.109747 21485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:57.131019 21485 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:41211
I20260809 03:26:57.131682 21622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:41211 every 8 connection(s)
I20260809 03:26:57.132215 21485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:26:57.136723 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 21485
I20260809 03:26:57.136878 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:26:57.137192 9403 external_mini_cluster.cc:1428] Setting key 445d75dcb45fdc3e8e6018e505eb6502
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:26:57.149262 21626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:57.136082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:36791 (local address 127.9.46.254:46197)
0809 03:26:57.136358 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:57.136362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:57.136378 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:26:57.137615 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:57.137620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:57.137685 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:57.137773 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:57.139837 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.140724 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.141987 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.142238 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.142357 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:57.145099 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:57.145108 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:57.145111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:26:57.145135 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:57.147034 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:57.147588 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:57.147591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:57.147593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:57.147681 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:57.148035 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:57.148040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:57.148041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:57.148231 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:57.148343 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:57.148939 (+ 596us) server_negotiation.cc:299] Negotiation successful
0809 03:26:57.149072 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":122,"threads_started":1}
I20260809 03:26:57.150207 21625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:57.135021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46197 (local address 127.9.46.193:36791)
0809 03:26:57.135781 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:57.135845 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:57.137304 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:57.138040 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:57.138051 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:57.138636 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:57.139621 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:57.139643 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.140896 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.140898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:57.141853 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:57.141862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.142400 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.142406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:57.142587 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:57.143270 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:57.143287 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:57.144901 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:57.147176 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:57.147182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:57.147198 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:57.147469 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:57.147779 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:57.147783 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:57.147785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:57.147936 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:57.148354 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:57.148363 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:57.148691 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 03:26:57.148885 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 03:26:57.149214 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":252,"threads_started":1}
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:26:57.151558 21623 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46197
I20260809 03:26:57.151845 21623 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:57.152462 21623 heartbeater.cc:511] Master 127.9.46.254:46197 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
I20260809 03:26:57.154281 21399 ts_manager.cc:194] Registered new tserver with Master: 83ecdf4b1df34748a16614b644b2e94d (127.9.46.193:41211)
I20260809 03:26:57.155866 21399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:36791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:26:57.173854 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:54567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46197
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 03:26:57.375881 21629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:57.376273 21629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:57.376333 21629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:57.383389 21629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:57.383488 21629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:57.383590 21629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:26:57.392755 21629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.46.212:54567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:57.394181 21629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:57.395673 21629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:57.409960 21638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:57.410036 21637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:57.410419 21640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:57.411413 21629 server_base.cc:1034] running on GCE node
I20260809 03:26:57.411999 21629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:57.412844 21629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:57.416128 21629 hybrid_clock.cc:648] HybridClock initialized: now 1786246017416094 us; error 56 us; skew 500 ppm
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:26:57.421048 21629 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:26:57.422248 21629 webserver.cc:460] Webserver started at http://127.9.46.194:41427/ using document root <none> and password file <none>
I20260809 03:26:57.422993 21629 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:57.423084 21629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:57.423323 21629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:26:57.448608 21629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "8fadc0c659204ccb8f78f0c7a7209d43"
format_stamp: "Formatted at 2026-08-09 03:26:57 on dist-test-slave-65mx"
server_key: "2bd529339cf9b594e82c5b5006ed81f4"
server_key_iv: "316dcaaa9ab0def6cdaf08513eaefb35"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:57.449520 21629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "8fadc0c659204ccb8f78f0c7a7209d43"
format_stamp: "Formatted at 2026-08-09 03:26:57 on dist-test-slave-65mx"
server_key: "2bd529339cf9b594e82c5b5006ed81f4"
server_key_iv: "316dcaaa9ab0def6cdaf08513eaefb35"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:57.455374 21629 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260809 03:26:57.480239 21650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:57.482177 21629 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.001s
I20260809 03:26:57.482360 21629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "8fadc0c659204ccb8f78f0c7a7209d43"
format_stamp: "Formatted at 2026-08-09 03:26:57 on dist-test-slave-65mx"
server_key: "2bd529339cf9b594e82c5b5006ed81f4"
server_key_iv: "316dcaaa9ab0def6cdaf08513eaefb35"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:57.482537 21629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:57.504810 21629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:57.508700 21629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:57.508899 21629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:57.509593 21629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:57.510651 21629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:57.510704 21629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:57.510774 21629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:57.510799 21629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:57.530313 21629 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:36421
I20260809 03:26:57.531805 21629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:26:57.535840 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 21629
I20260809 03:26:57.536036 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:26:57.536434 9403 external_mini_cluster.cc:1428] Setting key 01ff0319b6d39fbec206717a2cc7abde
I20260809 03:26:57.542511 21777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:36421 every 8 connection(s)
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:26:57.568909 21626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:57.548754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:40157 (local address 127.9.46.254:46197)
0809 03:26:57.548886 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:57.548891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:57.548909 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:26:57.549698 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:57.549703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:57.549762 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:57.549823 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:57.551666 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.552657 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.555436 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.555896 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.556044 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:57.560875 (+ 4831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:57.560880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:57.560883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:26:57.560920 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:57.563338 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:57.566630 (+ 3292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:57.566634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:57.566639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:57.566788 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:57.567326 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:57.567331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:57.567339 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:57.567596 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:57.567701 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:57.568525 (+ 824us) server_negotiation.cc:299] Negotiation successful
0809 03:26:57.568725 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:26:57.570935 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:57.547631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46197 (local address 127.9.46.194:40157)
0809 03:26:57.548220 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:57.548283 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:57.549422 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:57.550277 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:57.550288 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:57.550691 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:57.551452 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:57.551476 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.553126 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.553130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:57.555250 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:57.555263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.556298 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.556302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:57.556605 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:57.557628 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:57.557652 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:57.560502 (+ 2850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:57.563518 (+ 3016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:57.563525 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:57.563543 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:57.564080 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:57.566933 (+ 2853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:57.566937 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:57.566939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:57.567197 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:57.567779 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:57.567787 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:57.568200 (+ 413us) client_negotiation.cc:769] Sending connection context
0809 03:26:57.569445 (+ 1245us) client_negotiation.cc:240] Negotiation successful
0809 03:26:57.569838 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":177,"threads_started":1}
I20260809 03:26:57.578819 21778 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46197
I20260809 03:26:57.579113 21778 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:57.579814 21778 heartbeater.cc:511] Master 127.9.46.254:46197 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:26:57.581118 21399 ts_manager.cc:194] Registered new tserver with Master: 8fadc0c659204ccb8f78f0c7a7209d43 (127.9.46.194:36421)
I20260809 03:26:57.581959 21399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:40157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:26:57.612792 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:54567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46197
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:26:57.798386 21783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:26:57.798939 21783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:26:57.799080 21783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:26:57.805779 21783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:26:57.805927 21783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:26:57.806090 21783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:26:57.814313 21783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.46.212:54567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:26:57.815943 21783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:26:57.817408 21783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:26:57.831948 21790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:26:57.833066 21789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:57.833321 21783 server_base.cc:1034] running on GCE node
W20260809 03:26:57.834127 21792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:26:57.834682 21783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:26:57.835482 21783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:26:57.836649 21783 hybrid_clock.cc:648] HybridClock initialized: now 1786246017836585 us; error 86 us; skew 500 ppm
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:26:57.841367 21783 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:26:57.842553 21783 webserver.cc:460] Webserver started at http://127.9.46.195:45433/ using document root <none> and password file <none>
I20260809 03:26:57.843323 21783 fs_manager.cc:359] Metadata directory not provided
I20260809 03:26:57.843499 21783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:26:57.843876 21783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:26:57.864836 21783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "2ea38e056dba46d790b3b4eef9c0d099"
format_stamp: "Formatted at 2026-08-09 03:26:57 on dist-test-slave-65mx"
server_key: "7234858c9aab0a3683e81b7d8bbb2300"
server_key_iv: "789dea1f6ab6e3801c06cb7fd69b95e3"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:57.865895 21783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "2ea38e056dba46d790b3b4eef9c0d099"
format_stamp: "Formatted at 2026-08-09 03:26:57 on dist-test-slave-65mx"
server_key: "7234858c9aab0a3683e81b7d8bbb2300"
server_key_iv: "789dea1f6ab6e3801c06cb7fd69b95e3"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:57.872059 21783 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 03:26:57.900460 21799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:57.902375 21783 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:26:57.902518 21783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "2ea38e056dba46d790b3b4eef9c0d099"
format_stamp: "Formatted at 2026-08-09 03:26:57 on dist-test-slave-65mx"
server_key: "7234858c9aab0a3683e81b7d8bbb2300"
server_key_iv: "789dea1f6ab6e3801c06cb7fd69b95e3"
server_key_version: "kuduclusterkey@0"
I20260809 03:26:57.902648 21783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:26:57.927661 21783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:26:57.931850 21783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:26:57.932122 21783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:26:57.932960 21783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:26:57.934095 21783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:26:57.934204 21783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:57.934300 21783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:26:57.934362 21783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:26:57.951198 21783 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:41051
I20260809 03:26:57.952517 21783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:26:57.959177 21920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:41051 every 8 connection(s)
I20260809 03:26:57.961170 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 21783
I20260809 03:26:57.961329 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:26:57.961683 9403 external_mini_cluster.cc:1428] Setting key 581eafa6b081201ca9c23157a191092a
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
Aug 09 03:26:57 dist-test-slave-65mx krb5kdc[20873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:26:57.980432 21626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:57.964263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:49089 (local address 127.9.46.254:46197)
0809 03:26:57.964402 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:57.964407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:57.968119 (+ 3712us) server_negotiation.cc:407] Connection header received
0809 03:26:57.969456 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:57.969459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:57.969514 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:57.969592 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:57.970736 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.971384 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.973156 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.973383 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.973495 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:57.976405 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:57.976409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:57.976411 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:26:57.976439 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:57.978236 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:57.978858 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:57.978861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:57.978863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:57.978951 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:57.979305 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:57.979309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:57.979311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:57.979500 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:57.979610 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:57.980142 (+ 532us) server_negotiation.cc:299] Negotiation successful
0809 03:26:57.980262 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:26:57.981438 21923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:57.962958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46197 (local address 127.9.46.195:49089)
0809 03:26:57.967919 (+ 4961us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:57.967990 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:57.969194 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:57.969773 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:57.969784 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:57.970078 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:26:57.970588 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:57.970602 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.971690 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.971693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:57.973014 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:57.973024 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.973539 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.973543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:57.973722 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:57.974336 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:26:57.974353 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:26:57.976238 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:26:57.978373 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:57.978378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:57.978391 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:57.978730 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:57.979047 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:26:57.979051 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:26:57.979053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:26:57.979207 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:26:57.979616 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:26:57.979625 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:26:57.979888 (+ 263us) client_negotiation.cc:769] Sending connection context
0809 03:26:57.980080 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 03:26:57.980402 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4702,"thread_start_us":160,"threads_started":1}
I20260809 03:26:57.982517 21921 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46197
I20260809 03:26:57.982755 21921 heartbeater.cc:464] Registering TS with master...
I20260809 03:26:57.983280 21921 heartbeater.cc:511] Master 127.9.46.254:46197 requested a full tablet report, sending...
I20260809 03:26:57.984278 21399 ts_manager.cc:194] Registered new tserver with Master: 2ea38e056dba46d790b3b4eef9c0d099 (127.9.46.195:41051)
I20260809 03:26:57.984963 21399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:49089
I20260809 03:26:57.990316 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:26:58.003612 21626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:57.992989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52790 (local address 127.9.46.254:46197)
0809 03:26:57.993117 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:57.993122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:57.993430 (+ 308us) server_negotiation.cc:407] Connection header received
0809 03:26:57.993614 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:57.993617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:57.993662 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:57.993712 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:26:57.994705 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.995511 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.997032 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:57.997407 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:57.997506 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:57.999618 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:26:57.999622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:26:57.999625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:26:57.999653 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:26:58.001687 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:58.002206 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:58.002210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:58.002211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:58.002299 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:26:58.002622 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:26:58.002625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:26:58.002627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:26:58.002850 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:26:58.002956 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:58.003337 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 03:26:58.003454 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:26:58.007651 21399 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:26:58.014006 21399 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:26:58.034263 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.024964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:41051 (local address 127.0.0.1:40844)
0809 03:26:58.027387 (+ 2423us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:58.027401 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:58.027523 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:58.028312 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:58.028318 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:58.028342 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:26:58.028622 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:58.028631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.030313 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.030317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:58.031868 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:58.031879 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.033699 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.033702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:58.033879 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.033916 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 03:26:58.034014 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 03:26:58.034125 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2362,"thread_start_us":79,"threads_started":1}
I20260809 03:26:58.034914 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.025651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.9.46.195:41051)
0809 03:26:58.027186 (+ 1535us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:58.027218 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:58.027634 (+ 416us) server_negotiation.cc:407] Connection header received
0809 03:26:58.027724 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:58.027729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:58.027940 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:58.028119 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:26:58.028762 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.030176 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.032121 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.033007 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.033115 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.033263 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:58.034601 (+ 1338us) server_negotiation.cc:299] Negotiation successful
0809 03:26:58.034716 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1425,"thread_start_us":141,"threads_started":1}
I20260809 03:26:58.038661 21852 tablet_service.cc:1467] Processing CreateTablet for tablet 05361fc97fee42029c032ebb658280d8 (DEFAULT_TABLE table=test-table [id=428a05fb4a9f4941b0b661f7bff8e7cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:26:58.040335 21852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05361fc97fee42029c032ebb658280d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:26:58.052927 21940 tablet_bootstrap.cc:492] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: Bootstrap starting.
I20260809 03:26:58.054476 21935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.024833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:36421 (local address 127.0.0.1:38068)
0809 03:26:58.026588 (+ 1755us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:58.026654 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:58.026781 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:58.028501 (+ 1720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:58.028504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:58.028539 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:26:58.029540 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:58.029548 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.046614 (+ 17066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.046619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:58.048920 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:58.048932 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.053154 (+ 4222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.053160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:58.053353 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.053383 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 03:26:58.054252 (+ 869us) client_negotiation.cc:240] Negotiation successful
0809 03:26:58.054339 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1572,"thread_start_us":102,"threads_started":1}
I20260809 03:26:58.055449 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.025644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38068 (local address 127.9.46.194:36421)
0809 03:26:58.027626 (+ 1982us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:58.027643 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:58.027664 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:26:58.027726 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:58.027730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:58.027940 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:58.028348 (+ 408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:26:58.029683 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.045682 (+ 15999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.049209 (+ 3527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.050219 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.054850 (+ 4631us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.055009 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:58.055061 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 03:26:58.055182 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1867,"thread_start_us":76,"threads_started":1}
I20260809 03:26:58.057741 21938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.026729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:41211 (local address 127.0.0.1:46204)
0809 03:26:58.033461 (+ 6732us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:58.033476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:58.033589 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:58.051041 (+ 17452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:58.051045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:58.051063 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:26:58.051157 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:58.051162 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.054766 (+ 3604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.054769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:58.056093 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:58.056103 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.057308 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.057314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:58.057401 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.057483 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 03:26:58.057536 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 03:26:58.057602 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6667,"thread_start_us":112,"threads_started":1}
I20260809 03:26:58.058647 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.027184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.9.46.193:41211)
0809 03:26:58.050590 (+ 23406us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:58.050597 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:58.050632 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:26:58.050705 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:58.050710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:58.050893 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:58.051294 (+ 401us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:26:58.051422 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.052536 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.056255 (+ 3719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.057150 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.058178 (+ 1028us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.058325 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:58.058377 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 03:26:58.058468 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23291,"thread_start_us":100,"threads_started":1}
I20260809 03:26:58.058835 21706 tablet_service.cc:1467] Processing CreateTablet for tablet 05361fc97fee42029c032ebb658280d8 (DEFAULT_TABLE table=test-table [id=428a05fb4a9f4941b0b661f7bff8e7cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:26:58.059981 21706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05361fc97fee42029c032ebb658280d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:26:58.062475 21551 tablet_service.cc:1467] Processing CreateTablet for tablet 05361fc97fee42029c032ebb658280d8 (DEFAULT_TABLE table=test-table [id=428a05fb4a9f4941b0b661f7bff8e7cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:26:58.063683 21551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05361fc97fee42029c032ebb658280d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:26:58.073835 21940 tablet_bootstrap.cc:654] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: Neither blocks nor log segments found. Creating new log.
I20260809 03:26:58.074836 21940 log.cc:824] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: Log is configured to *not* fsync() on all Append() calls
I20260809 03:26:58.075832 21941 tablet_bootstrap.cc:492] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d: Bootstrap starting.
I20260809 03:26:58.078245 21940 tablet_bootstrap.cc:492] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: No bootstrap required, opened a new log
I20260809 03:26:58.078595 21940 ts_tablet_manager.cc:1397] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.000s
I20260809 03:26:58.078627 21941 tablet_bootstrap.cc:654] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d: Neither blocks nor log segments found. Creating new log.
I20260809 03:26:58.080039 21941 log.cc:824] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:26:58.082697 21941 tablet_bootstrap.cc:492] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d: No bootstrap required, opened a new log
I20260809 03:26:58.082917 21941 ts_tablet_manager.cc:1397] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260809 03:26:58.084954 21942 tablet_bootstrap.cc:492] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: Bootstrap starting.
I20260809 03:26:58.086061 21940 raft_consensus.cc:348] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 } } peers { permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 } } peers { permanent_uuid: "83ecdf4b1df34748a16614b644b2e94d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 41211 } }
I20260809 03:26:58.087721 21940 raft_consensus.cc:374] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:26:58.087832 21940 raft_consensus.cc:729] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea38e056dba46d790b3b4eef9c0d099, State: Initialized, Role: FOLLOWER
I20260809 03:26:58.088836 21941 raft_consensus.cc:348] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 } } peers { permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 } } peers { permanent_uuid: "83ecdf4b1df34748a16614b644b2e94d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 41211 } }
I20260809 03:26:58.089188 21941 raft_consensus.cc:374] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:26:58.089157 21940 consensus_queue.cc:260] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 } } peers { permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 } } peers { permanent_uuid: "83ecdf4b1df34748a16614b644b2e94d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 41211 } }
I20260809 03:26:58.089404 21941 raft_consensus.cc:729] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ecdf4b1df34748a16614b644b2e94d, State: Initialized, Role: FOLLOWER
I20260809 03:26:58.089939 21941 consensus_queue.cc:260] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 } } peers { permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 } } peers { permanent_uuid: "83ecdf4b1df34748a16614b644b2e94d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 41211 } }
I20260809 03:26:58.090360 21940 ts_tablet_manager.cc:1428] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260809 03:26:58.090950 21941 ts_tablet_manager.cc:1428] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 03:26:58.091353 21623 heartbeater.cc:503] Master 127.9.46.254:46197 was elected leader, sending a full tablet report...
I20260809 03:26:58.092401 21921 heartbeater.cc:503] Master 127.9.46.254:46197 was elected leader, sending a full tablet report...
I20260809 03:26:58.093824 21942 tablet_bootstrap.cc:654] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: Neither blocks nor log segments found. Creating new log.
I20260809 03:26:58.095619 21942 log.cc:824] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: Log is configured to *not* fsync() on all Append() calls
I20260809 03:26:58.098042 21942 tablet_bootstrap.cc:492] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: No bootstrap required, opened a new log
I20260809 03:26:58.098248 21942 ts_tablet_manager.cc:1397] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260809 03:26:58.102666 21942 raft_consensus.cc:348] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 } } peers { permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 } } peers { permanent_uuid: "83ecdf4b1df34748a16614b644b2e94d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 41211 } }
I20260809 03:26:58.102926 21942 raft_consensus.cc:374] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:26:58.103000 21942 raft_consensus.cc:729] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fadc0c659204ccb8f78f0c7a7209d43, State: Initialized, Role: FOLLOWER
I20260809 03:26:58.103447 21942 consensus_queue.cc:260] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 } } peers { permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 } } peers { permanent_uuid: "83ecdf4b1df34748a16614b644b2e94d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 41211 } }
I20260809 03:26:58.104069 21778 heartbeater.cc:503] Master 127.9.46.254:46197 was elected leader, sending a full tablet report...
I20260809 03:26:58.104579 21942 ts_tablet_manager.cc:1428] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260809 03:26:58.135129 21624 tablet.cc:2369] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:26:58.212075 21922 tablet.cc:2369] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:26:58.223513 21946 raft_consensus.cc:482] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:26:58.223798 21946 raft_consensus.cc:504] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 } } peers { permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 } } peers { permanent_uuid: "83ecdf4b1df34748a16614b644b2e94d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 41211 } }
I20260809 03:26:58.225286 21946 leader_election.cc:288] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fadc0c659204ccb8f78f0c7a7209d43 (127.9.46.194:36421), 2ea38e056dba46d790b3b4eef9c0d099 (127.9.46.195:41051)
I20260809 03:26:58.231172 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.226017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:53419 (local address 127.9.46.194:36421)
0809 03:26:58.226169 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:58.226175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:58.226468 (+ 293us) server_negotiation.cc:407] Connection header received
0809 03:26:58.226682 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:58.226686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:58.226736 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:58.226784 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:26:58.227280 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.228080 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.229810 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.230406 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.230518 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.230571 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:58.230949 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 03:26:58.231042 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:26:58.231244 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.226097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:39079 (local address 127.9.46.195:41051)
0809 03:26:58.226218 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:58.226223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:58.226598 (+ 375us) server_negotiation.cc:407] Connection header received
0809 03:26:58.226815 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:58.226818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:58.226870 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:58.226923 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:26:58.227428 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.228193 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.229943 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.230550 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.230645 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.230691 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:58.231042 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 03:26:58.231120 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:26:58.231655 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.225547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:36421 (local address 127.9.46.193:53419)
0809 03:26:58.226357 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:58.226373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:58.226557 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:58.226957 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:58.226960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:58.226979 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:26:58.227132 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:58.227141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.228235 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.228238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:58.229660 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:58.229671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.230694 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.230697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:58.230783 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.230816 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:26:58.231383 (+ 567us) client_negotiation.cc:240] Negotiation successful
0809 03:26:58.231493 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":21,"thread_start_us":121,"threads_started":1}
I20260809 03:26:58.232147 21951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.225630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:41051 (local address 127.9.46.193:39079)
0809 03:26:58.226430 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:58.226444 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:26:58.226682 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:26:58.227071 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:26:58.227074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:26:58.227095 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:26:58.227277 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:58.227286 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.228463 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.228466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:58.229797 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:26:58.229808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.230787 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.230790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:26:58.230895 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.230916 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:26:58.231798 (+ 882us) client_negotiation.cc:240] Negotiation successful
0809 03:26:58.231879 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":171,"threads_started":1}
I20260809 03:26:58.232581 21716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05361fc97fee42029c032ebb658280d8" candidate_uuid: "83ecdf4b1df34748a16614b644b2e94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" is_pre_election: true
I20260809 03:26:58.232975 21716 raft_consensus.cc:2454] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ecdf4b1df34748a16614b644b2e94d in term 0.
I20260809 03:26:58.233083 21873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05361fc97fee42029c032ebb658280d8" candidate_uuid: "83ecdf4b1df34748a16614b644b2e94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea38e056dba46d790b3b4eef9c0d099" is_pre_election: true
I20260809 03:26:58.233407 21873 raft_consensus.cc:2454] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ecdf4b1df34748a16614b644b2e94d in term 0.
I20260809 03:26:58.233592 21508 leader_election.cc:302] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ecdf4b1df34748a16614b644b2e94d, 8fadc0c659204ccb8f78f0c7a7209d43; no voters:
I20260809 03:26:58.234048 21946 raft_consensus.cc:2781] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:26:58.234169 21946 raft_consensus.cc:482] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:26:58.234241 21946 raft_consensus.cc:3037] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:26:58.235383 21946 raft_consensus.cc:504] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 } } peers { permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 } } peers { permanent_uuid: "83ecdf4b1df34748a16614b644b2e94d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 41211 } }
I20260809 03:26:58.235783 21946 leader_election.cc:288] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [CANDIDATE]: Term 1 election: Requested vote from peers 8fadc0c659204ccb8f78f0c7a7209d43 (127.9.46.194:36421), 2ea38e056dba46d790b3b4eef9c0d099 (127.9.46.195:41051)
I20260809 03:26:58.236374 21873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05361fc97fee42029c032ebb658280d8" candidate_uuid: "83ecdf4b1df34748a16614b644b2e94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea38e056dba46d790b3b4eef9c0d099"
I20260809 03:26:58.236361 21716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05361fc97fee42029c032ebb658280d8" candidate_uuid: "83ecdf4b1df34748a16614b644b2e94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fadc0c659204ccb8f78f0c7a7209d43"
I20260809 03:26:58.236506 21873 raft_consensus.cc:3037] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:26:58.236506 21716 raft_consensus.cc:3037] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:26:58.238306 21873 raft_consensus.cc:2454] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ecdf4b1df34748a16614b644b2e94d in term 1.
I20260809 03:26:58.238306 21716 raft_consensus.cc:2454] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ecdf4b1df34748a16614b644b2e94d in term 1.
I20260809 03:26:58.238790 21508 leader_election.cc:302] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ecdf4b1df34748a16614b644b2e94d, 8fadc0c659204ccb8f78f0c7a7209d43; no voters:
I20260809 03:26:58.238981 21946 raft_consensus.cc:2781] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:26:58.239246 21946 raft_consensus.cc:686] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [term 1 LEADER]: Becoming Leader. State: Replica: 83ecdf4b1df34748a16614b644b2e94d, State: Running, Role: LEADER
I20260809 03:26:58.239595 21946 consensus_queue.cc:237] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 } } peers { permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 } } peers { permanent_uuid: "83ecdf4b1df34748a16614b644b2e94d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 41211 } }
I20260809 03:26:58.242981 21399 catalog_manager.cc:5495] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ecdf4b1df34748a16614b644b2e94d (127.9.46.193). New cstate: current_term: 1 leader_uuid: "83ecdf4b1df34748a16614b644b2e94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83ecdf4b1df34748a16614b644b2e94d" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 41211 } health_report { overall_health: HEALTHY } } }
I20260809 03:26:58.274376 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.270700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46214 (local address 127.9.46.193:41211)
0809 03:26:58.270836 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:26:58.270841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:26:58.270854 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:26:58.270939 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:26:58.270941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:26:58.270993 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:26:58.271108 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:26:58.271373 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.271881 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.272827 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:26:58.273047 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:26:58.273162 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:26:58.273467 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:26:58.273884 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:26:58.274019 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 03:26:58.274081 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 03:26:58.274139 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:26:58.281826 21873 raft_consensus.cc:1261] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099 [term 1 FOLLOWER]: Refusing update from remote peer 83ecdf4b1df34748a16614b644b2e94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:26:58.283524 21952 consensus_queue.cc:1035] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea38e056dba46d790b3b4eef9c0d099" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:26:58.285394 21716 raft_consensus.cc:1261] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43 [term 1 FOLLOWER]: Refusing update from remote peer 83ecdf4b1df34748a16614b644b2e94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:26:58.286237 21946 consensus_queue.cc:1035] T 05361fc97fee42029c032ebb658280d8 P 83ecdf4b1df34748a16614b644b2e94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fadc0c659204ccb8f78f0c7a7209d43" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:26:58.300416 21779 tablet.cc:2369] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:26:58.307193 21955 mvcc.cc:205] Tried to move back new op lower bound from 7316463690871058432 to 7316463690712379392. Current Snapshot: MvccSnapshot[applied={T|T < 7316463690871058432 or (T in {7316463690871058432})}]
I20260809 03:26:58.307811 21957 mvcc.cc:205] Tried to move back new op lower bound from 7316463690871058432 to 7316463690712379392. Current Snapshot: MvccSnapshot[applied={T|T < 7316463690871058432}]
I20260809 03:26:58.310374 21956 mvcc.cc:205] Tried to move back new op lower bound from 7316463690871058432 to 7316463690712379392. Current Snapshot: MvccSnapshot[applied={T|T < 7316463690871058432}]
I20260809 03:26:58.314052 21398 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:26:58.314239 21398 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:26:58.316882 21398 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d314d05d3ec4a91ba66a60504fe7d33: Sending DeleteTablet for 3 replicas of tablet 05361fc97fee42029c032ebb658280d8
I20260809 03:26:58.318116 21706 tablet_service.cc:1514] Processing DeleteTablet for tablet 05361fc97fee42029c032ebb658280d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:26:58 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:38068
I20260809 03:26:58.318643 21551 tablet_service.cc:1514] Processing DeleteTablet for tablet 05361fc97fee42029c032ebb658280d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:26:58 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:46204
I20260809 03:26:58.319603 21852 tablet_service.cc:1514] Processing DeleteTablet for tablet 05361fc97fee42029c032ebb658280d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:26:58 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:40844
I20260809 03:26:58.320942 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 21485
I20260809 03:26:58.321761 21964 tablet_replica.cc:330] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: stopping tablet replica
I20260809 03:26:58.321943 21966 tablet_replica.cc:330] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: stopping tablet replica
I20260809 03:26:58.322137 21964 raft_consensus.cc:2229] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:26:58.322212 21966 raft_consensus.cc:2229] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:26:58.322454 21966 raft_consensus.cc:2258] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:26:58.322454 21964 raft_consensus.cc:2258] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:26:58.324136 21966 ts_tablet_manager.cc:1905] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:26:58.327306 21966 ts_tablet_manager.cc:1918] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:26:58.327447 21966 log.cc:1196] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/05361fc97fee42029c032ebb658280d8
I20260809 03:26:58.327832 21966 ts_tablet_manager.cc:1939] T 05361fc97fee42029c032ebb658280d8 P 2ea38e056dba46d790b3b4eef9c0d099: Deleting consensus metadata
I20260809 03:26:58.329389 21383 catalog_manager.cc:4841] TS 2ea38e056dba46d790b3b4eef9c0d099 (127.9.46.195:41051): tablet 05361fc97fee42029c032ebb658280d8 (table test-table [id=428a05fb4a9f4941b0b661f7bff8e7cb]) successfully deleted
I20260809 03:26:58.334164 21964 ts_tablet_manager.cc:1905] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 03:26:58.336900 21385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:41211 (error 108)
I20260809 03:26:58.337188 21964 ts_tablet_manager.cc:1918] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:26:58.337283 21964 log.cc:1196] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/05361fc97fee42029c032ebb658280d8
I20260809 03:26:58.337611 21964 ts_tablet_manager.cc:1939] T 05361fc97fee42029c032ebb658280d8 P 8fadc0c659204ccb8f78f0c7a7209d43: Deleting consensus metadata
I20260809 03:26:58.338801 21938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:26:58.338443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:41211 (local address 127.0.0.1:46218)
0809 03:26:58.338643 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 03:26:58.338714 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:41211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260809 03:26:58.339048 21385 catalog_manager.cc:4584] TS 83ecdf4b1df34748a16614b644b2e94d (127.9.46.193:41211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05361fc97fee42029c032ebb658280d8: Network error: Client connection negotiation failed: client connection to 127.9.46.193:41211: connect: Connection refused (error 111)
I20260809 03:26:58.339257 21385 catalog_manager.cc:4841] TS 8fadc0c659204ccb8f78f0c7a7209d43 (127.9.46.194:36421): tablet 05361fc97fee42029c032ebb658280d8 (table test-table [id=428a05fb4a9f4941b0b661f7bff8e7cb]) successfully deleted
I20260809 03:26:58.339632 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 21629
I20260809 03:26:58.351675 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 21783
I20260809 03:26:58.364609 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 21365
I20260809 03:26:58.371846 9403 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 03:26:58.717607 9403 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 03:26:58.717775 9403 mini_ranger.cc:66] Stopping Ranger...
I20260809 03:26:59.062916 9403 mini_ranger.cc:68] Stopped Ranger
2026-08-09 03:26:59.063 UTC [20904] LOG: received smart shutdown request
2026-08-09 03:26:59.066 UTC [20904] LOG: background worker "logical replication launcher" (PID 20915) exited with exit code 1
2026-08-09 03:26:59.067 UTC [20910] LOG: shutting down
2026-08-09 03:26:59.244 UTC [20904] LOG: database system is shut down
2026-08-09T03:26:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48436 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:26:59 dist-test-slave-65mx krb5kdc[21982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:26:59 dist-test-slave-65mx krb5kdc[21982](info): set up 2 sockets
Aug 09 03:26:59 dist-test-slave-65mx krb5kdc[21982](info): commencing operation
krb5kdc: starting...
W20260809 03:27:00.465359 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:27:00 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:27:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:27:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.46.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.46.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/rangeradmin_127.9.46.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/rangeradmin_127.9.46.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.46.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.46.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/rangerlookup_127.9.46.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/rangerlookup_127.9.46.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.46.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.46.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/HTTP_127.9.46.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/HTTP_127.9.46.212@KRBTEST.COM.keytab.
I20260809 03:27:00.648241 9403 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOU4Lix/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/postgres -l logfile start
2026-08-09 03:27:03.236 UTC [22015] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 03:27:03.236 UTC [22015] LOG: listening on IPv4 address "127.9.46.212", port 48071
2026-08-09 03:27:03.240 UTC [22015] LOG: listening on Unix socket "/tmp/.s.PGSQL.48071"
2026-08-09 03:27:03.262 UTC [22018] LOG: database system was shut down at 2026-08-09 03:27:01 UTC
2026-08-09 03:27:03.266 UTC [22015] LOG: database system is ready to accept connections
I20260809 03:27:04.265429 9403 mini_postgres.cc:96] Postgres bound to 48071
2026-08-09 03:27:04.272 UTC [22027] FATAL: database "slave" does not exist
127.9.46.212:48071 - accepting connections
I20260809 03:27:04.273360 9403 mini_ranger.cc:161] Starting Ranger...
I20260809 03:27:04.291230 9403 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 03:27:04.768153 9403 mini_ranger.cc:87] Created ranger Postgres database
I20260809 03:27:04.768244 9403 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-admin
2026-08-09 03:27:05,041 [I] DB FLAVOR :POSTGRES
2026-08-09 03:27:05,041 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:27:05,041 [I] Checking connection..
2026-08-09 03:27:05,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 03:27:05,313 [I] Checking connection passed.
2026-08-09 03:27:05,313 [I] --------- Verifying version history table ---------
2026-08-09 03:27:05,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:27:05,567 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:27:05,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:27:05,830 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 03:27:05,830 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 03:27:05,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 03:27:06.100 UTC [22097] WARNING: there is no transaction in progress
2026-08-09 03:27:06,108 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 03:27:06,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 03:27:06,374 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 03:27:06,374 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:27:06,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 03:27:06,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 03:27:06,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-65mx.c.gcp-upstream.internal','N') ;"
2026-08-09 03:27:07,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:27:07,419 [I] Table x_portal_user does not exist in database ranger
2026-08-09 03:27:07,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:27:07,683 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 03:27:07,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:27:07,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:27:08,193 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 03:27:08,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 03:27:09.164 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.180 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.197 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.212 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.228 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.245 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.300 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.307 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.321 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.332 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.343 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.357 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.368 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.374 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.381 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:09.388 UTC [22214] WARNING: there is no transaction in progress
2026-08-09 03:27:10,054 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 03:27:10,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 03:27:10,287 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 03:27:10,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 03:27:10,533 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 03:27:10,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 03:27:10,803 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 03:27:10,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 03:27:11,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 03:27:11,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-65mx.c.gcp-upstream.internal';"
2026-08-09 03:27:11,550 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 03:27:11,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 03:27:11,810 [I] DB_PATCHES have already been applied
I20260809 03:27:11.816059 9403 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOU4Lix/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 03:27:11.816123 9403 mini_ranger.cc:193] Using host: 127.9.46.212
I20260809 03:27:11.820214 9403 mini_ranger.cc:227] Ranger admin URL: http://127.9.46.212:60005
Aug 09, 2026 3:27:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:27:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:27:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:27:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:27:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:27:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:27:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 03:27:13.000679 9403 mini_ranger.cc:160] Time spent starting Ranger: real 8.727s user 0.000s sys 0.012s
Aug 09, 2026 3:27:32 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:27:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 3:27:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:27:32 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:27:33 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 03:27:34.372752 9403 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.46.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.46.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/rangerkms_127.9.46.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.46.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/rangerkms_127.9.46.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.46.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.46.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.46.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/HTTP_127.9.46.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.46.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/HTTP_127.9.46.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 03:27:34 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246054, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 03:27:34.484447 9403 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 03:27:34.496222 9403 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 03:27:35.539382 9403 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 03:27:35.539470 9403 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-kms
I20260809 03:27:35.539492 9403 mini_ranger_kms.cc:227] Using postgres at 127.9.46.212:48071
2026-08-09 03:27:35,720 [I] DB FLAVOR :POSTGRES
2026-08-09 03:27:35,720 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 03:27:35,720 [I] Checking connection
2026-08-09 03:27:35,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 03:27:35,957 [I] connection success
2026-08-09 03:27:35,958 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 03:27:35,958 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 03:27:35,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 03:27:36,223 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 03:27:36,223 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 03:27:36,223 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 03:27:36,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.46.212:48071/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 03:27:36,507 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 03:27:37.024996 9403 mini_ranger_kms.cc:310] Created kms service
I20260809 03:27:37.208297 9403 mini_ranger_kms.cc:326] Created kudu user
I20260809 03:27:37.259914 9403 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 03:27:37 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246054, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:37.667665 9403 mini_ranger_kms.cc:384] Added ranger policy
I20260809 03:27:37.667816 9403 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOU4Lix/build/debug/bin/postgresql.jar:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOU4Lix/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 03:27:37.667855 9403 mini_ranger_kms.cc:240] Using host: 127.9.46.212
I20260809 03:27:37.673213 9403 mini_ranger_kms.cc:276] Ranger KMS PID: 22415
I20260809 03:27:37.673317 9403 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.46.212:58315
Aug 09, 2026 3:27:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:27:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:27:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:27:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:27:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:27:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 03:27:40.083386 9403 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.599s user 0.018s sys 0.145s
I20260809 03:27:40.083544 9403 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 03:27:40.083622 9403 mini_ranger_kms.cc:401] 127.9.46.212:58315/kms/v1/keys
Aug 09 03:27:40 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246060, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.46.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 03:27:40 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246060, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.46.212@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
Aug 09 03:27:41 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246060, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.46.212@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
Aug 09 03:27:43 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:27:43.460111 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34551
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:34551
--ranger_config_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:58315/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:43.572995 22471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:43.573269 22471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:43.573308 22471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:43.577423 22471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:27:43.577482 22471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:43.577502 22471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:43.577517 22471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:27:43.577538 22471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:27:43.582983 22471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.46.212:58315/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:34551
--ranger_config_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:43.584043 22471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:43.585109 22471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:43.593858 22477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:43.593951 22471 server_base.cc:1034] running on GCE node
W20260809 03:27:43.593886 22476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:43.594143 22479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:43.594658 22471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:43.595767 22471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:43.596997 22471 hybrid_clock.cc:648] HybridClock initialized: now 1786246063596971 us; error 47 us; skew 500 ppm
Aug 09 03:27:43 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246063, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:43.599922 22471 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:27:43.601044 22471 webserver.cc:460] Webserver started at http://127.9.46.254:43009/ using document root <none> and password file <none>
I20260809 03:27:43.601603 22471 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:43.601666 22471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:43.601892 22471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:27:43 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246063, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:43.649942 22471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf"
format_stamp: "Formatted at 2026-08-09 03:27:43 on dist-test-slave-65mx"
server_key: "78233071dc244660443153f21c35689b"
server_key_iv: "d90d05cabcb300ad7ee377a204b35f64"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:43.650646 22471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf"
format_stamp: "Formatted at 2026-08-09 03:27:43 on dist-test-slave-65mx"
server_key: "78233071dc244660443153f21c35689b"
server_key_iv: "d90d05cabcb300ad7ee377a204b35f64"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:43.654803 22471 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:27:43.669435 22486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:43.670858 22471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:27:43.671010 22471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf"
format_stamp: "Formatted at 2026-08-09 03:27:43 on dist-test-slave-65mx"
server_key: "78233071dc244660443153f21c35689b"
server_key_iv: "d90d05cabcb300ad7ee377a204b35f64"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:43.671161 22471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:43.681766 22471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:43.684794 22471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:43.685002 22471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:43.693333 22471 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:34551
I20260809 03:27:43.693347 22545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:34551 every 8 connection(s)
I20260809 03:27:43.694423 22471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:27:43.697966 22547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:43.699056 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 22471
I20260809 03:27:43.699220 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:27:43.699538 9403 external_mini_cluster.cc:1428] Setting key 52091a5bf60e6c4a6e1b79d8361f42b1
I20260809 03:27:43.705250 22547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: Bootstrap starting.
Aug 09 03:27:43 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:43.708055 22547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:43.709036 22547 log.cc:824] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:43.711372 22547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: No bootstrap required, opened a new log
I20260809 03:27:43.713927 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:43.701014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.9.46.254:34551)
0809 03:27:43.701439 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:43.701454 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:43.701490 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:27:43.702113 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:43.702130 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:43.702446 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:43.702783 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:43.703589 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:43.704527 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:43.705492 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:43.705970 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:43.706146 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:43.708769 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:43.708775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:43.708787 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:43.708813 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:43.711152 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:43.711657 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:43.711663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:43.711674 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:43.711777 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:43.712070 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:43.712074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:43.712076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:43.712442 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:43.712667 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:43.712991 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 03:27:43.713286 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":135,"threads_started":1}
I20260809 03:27:43.715341 22547 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } }
I20260809 03:27:43.715624 22547 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:43.715713 22547 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2775fe1d0a04d25aa2c28aa1d17acaf, State: Initialized, Role: FOLLOWER
I20260809 03:27:43.716138 22547 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } }
I20260809 03:27:43.716308 22547 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:27:43.716374 22547 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:27:43.716466 22547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:43.717592 22547 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } }
I20260809 03:27:43.717941 22547 leader_election.cc:302] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2775fe1d0a04d25aa2c28aa1d17acaf; no voters:
I20260809 03:27:43.718293 22547 leader_election.cc:288] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:27:43.718386 22553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:27:43.718609 22553 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 1 LEADER]: Becoming Leader. State: Replica: f2775fe1d0a04d25aa2c28aa1d17acaf, State: Running, Role: LEADER
I20260809 03:27:43.718935 22553 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } }
I20260809 03:27:43.719527 22547 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:27:43.720566 22554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } } }
I20260809 03:27:43.720688 22554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:43.721060 22555 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2775fe1d0a04d25aa2c28aa1d17acaf. Latest consensus state: current_term: 1 leader_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } } }
I20260809 03:27:43.721212 22555 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:43.721942 22563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:27:43.722271 22547 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 03:27:43.728129 22563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:27:43.736505 22563 catalog_manager.cc:1347] Generated new cluster ID: 2402410c0384455baab97aaea4ac239b
I20260809 03:27:43.736608 22563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:27:43.767589 22563 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:27:43.769110 22563 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:27:43.777040 22563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: Generated new TSK 0
I20260809 03:27:43.777995 22563 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:27:44.472456 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:58315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34551
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:44.611315 22600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:44.611565 22600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:44.611609 22600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:44.615731 22600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:44.615792 22600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:44.615849 22600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:27:44.622278 22600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.46.212:58315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:44.623742 22600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:44.625348 22600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:44.636210 22607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:44.636214 22608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:44.636719 22610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:44.636881 22600 server_base.cc:1034] running on GCE node
I20260809 03:27:44.637315 22600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:44.638072 22600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:44.639297 22600 hybrid_clock.cc:648] HybridClock initialized: now 1786246064639239 us; error 76 us; skew 500 ppm
Aug 09 03:27:44 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:44.642163 22600 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:27:44.643112 22600 webserver.cc:460] Webserver started at http://127.9.46.193:42727/ using document root <none> and password file <none>
I20260809 03:27:44.643641 22600 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:44.643728 22600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:44.643977 22600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:27:44 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:44.658176 22600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "5659887875b9424a93ec34c72424b84a"
format_stamp: "Formatted at 2026-08-09 03:27:44 on dist-test-slave-65mx"
server_key: "4ee7fc5e31ba9c729b961d3c5d853f26"
server_key_iv: "0501f565fe970c6d4d0fff6a6b61fd94"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:44.659086 22600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "5659887875b9424a93ec34c72424b84a"
format_stamp: "Formatted at 2026-08-09 03:27:44 on dist-test-slave-65mx"
server_key: "4ee7fc5e31ba9c729b961d3c5d853f26"
server_key_iv: "0501f565fe970c6d4d0fff6a6b61fd94"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:44.666675 22600 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.003s
I20260809 03:27:44.683616 22617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:44.685379 22600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:27:44.685590 22600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "5659887875b9424a93ec34c72424b84a"
format_stamp: "Formatted at 2026-08-09 03:27:44 on dist-test-slave-65mx"
server_key: "4ee7fc5e31ba9c729b961d3c5d853f26"
server_key_iv: "0501f565fe970c6d4d0fff6a6b61fd94"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:44.685855 22600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:44.721477 22600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:44.725193 22600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:44.725432 22600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:44.726193 22600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:44.727252 22600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:44.727373 22600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:44.727504 22600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:44.727586 22600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:44.740854 22600 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:37317
I20260809 03:27:44.740935 22740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:37317 every 8 connection(s)
I20260809 03:27:44.742128 22600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:27:44.747548 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 22600
I20260809 03:27:44.747699 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:27:44.748019 9403 external_mini_cluster.cc:1428] Setting key 64cdd6741b90b658b1bc371677af150c
Aug 09 03:27:44 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:44.760977 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:44.744131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:43103 (local address 127.9.46.254:34551)
0809 03:27:44.746188 (+ 2057us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:44.746197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:44.746642 (+ 445us) server_negotiation.cc:407] Connection header received
0809 03:27:44.747918 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:44.747922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:44.747994 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:44.748080 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:44.749591 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:44.750469 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:44.752166 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:44.752530 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:44.752664 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:44.755769 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:44.755773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:44.755776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:44.755804 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:44.757780 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:44.758409 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:44.758413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:44.758415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:44.758508 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:44.758894 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:44.758897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:44.758898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:44.759133 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:44.759229 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:44.760657 (+ 1428us) server_negotiation.cc:299] Negotiation successful
0809 03:27:44.760789 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1962,"thread_start_us":105,"threads_started":1}
I20260809 03:27:44.762076 22744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:44.744587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34551 (local address 127.9.46.193:43103)
0809 03:27:44.746448 (+ 1861us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:44.746505 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:44.747574 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:44.748369 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:44.748378 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:44.748762 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:44.749414 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:44.749433 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:44.750623 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:44.750627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:44.752026 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:44.752037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:44.752840 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:44.752844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:44.753104 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:44.753793 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:44.753814 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:44.755568 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:44.757927 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:44.757933 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:44.757949 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:44.758284 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:44.758614 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:44.758617 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:44.758619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:44.758788 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:44.759284 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:44.759292 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:44.759592 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 03:27:44.759798 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 03:27:44.760136 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1647,"thread_start_us":141,"threads_started":1}
I20260809 03:27:44.765779 22741 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34551
I20260809 03:27:44.766021 22741 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:44.766556 22741 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
I20260809 03:27:44.768435 22504 ts_manager.cc:194] Registered new tserver with Master: 5659887875b9424a93ec34c72424b84a (127.9.46.193:37317)
I20260809 03:27:44.769982 22504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:43103
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:27:44.803676 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:58315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34551
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:27:44 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 03:27:44.955271 22747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:44.955621 22747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:44.955693 22747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:44.960206 22747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:44.960264 22747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:44.960350 22747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:27:44.967000 22747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.46.212:58315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:44.968137 22747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:44.969195 22747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:44.979841 22756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:44.979859 22753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:44.980297 22754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:44.981102 22747 server_base.cc:1034] running on GCE node
I20260809 03:27:44.981540 22747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:44.982275 22747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:44.985543 22747 hybrid_clock.cc:648] HybridClock initialized: now 1786246064985474 us; error 86 us; skew 500 ppm
Aug 09 03:27:44 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:44.989100 22747 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:27:44.990159 22747 webserver.cc:460] Webserver started at http://127.9.46.194:35357/ using document root <none> and password file <none>
I20260809 03:27:44.990854 22747 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:44.990953 22747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:44.991209 22747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:27:44 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:45.012408 22747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "f861eeb34d0e4f8487aaa634d9720316"
format_stamp: "Formatted at 2026-08-09 03:27:45 on dist-test-slave-65mx"
server_key: "73604bf283e9a198cb4359e9dbbbe99b"
server_key_iv: "0562200b91dc71c11facf73e17c933c0"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:45.014325 22747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "f861eeb34d0e4f8487aaa634d9720316"
format_stamp: "Formatted at 2026-08-09 03:27:45 on dist-test-slave-65mx"
server_key: "73604bf283e9a198cb4359e9dbbbe99b"
server_key_iv: "0562200b91dc71c11facf73e17c933c0"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:45.025810 22747 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.000s
I20260809 03:27:45.048615 22764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:45.050359 22747 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260809 03:27:45.050493 22747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "f861eeb34d0e4f8487aaa634d9720316"
format_stamp: "Formatted at 2026-08-09 03:27:45 on dist-test-slave-65mx"
server_key: "73604bf283e9a198cb4359e9dbbbe99b"
server_key_iv: "0562200b91dc71c11facf73e17c933c0"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:45.050637 22747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:45.066332 22747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:45.069991 22747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:45.070272 22747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:45.071007 22747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:45.072145 22747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:45.072260 22747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:45.072376 22747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:45.072446 22747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:45.086560 22747 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:42943
I20260809 03:27:45.087756 22747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:27:45.089798 22892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:42943 every 8 connection(s)
I20260809 03:27:45.091714 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 22747
I20260809 03:27:45.091856 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:27:45.092159 9403 external_mini_cluster.cc:1428] Setting key 594a61d8a9c38bb2e16973c3f191c3b1
Aug 09 03:27:45 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:45.108023 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:45.093263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:57709 (local address 127.9.46.254:34551)
0809 03:27:45.093417 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:45.093422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:45.094110 (+ 688us) server_negotiation.cc:407] Connection header received
0809 03:27:45.095458 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:45.095464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:45.095526 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:45.095586 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:45.097401 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:45.098217 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:45.099934 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:45.100274 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:45.100361 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:45.103851 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:45.103855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:45.103860 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:45.103891 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:45.105718 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:45.106306 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:45.106310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:45.106313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:45.106402 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:45.106813 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:45.106817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:45.106819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:45.107018 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:45.107136 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:45.107695 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 03:27:45.107834 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:27:45.109026 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:45.092145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34551 (local address 127.9.46.194:57709)
0809 03:27:45.093951 (+ 1806us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:45.094004 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:45.095218 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:45.096012 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:45.096021 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:45.096428 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:45.097231 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:45.097251 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:45.098386 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:45.098390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:45.099793 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:45.099803 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:45.100610 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:45.100615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:45.100828 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:45.101636 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:45.101662 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:45.103611 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:45.105869 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:45.105875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:45.105891 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:45.106181 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:45.106528 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:45.106533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:45.106535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:45.106696 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:45.107135 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:45.107143 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:45.107411 (+ 268us) client_negotiation.cc:769] Sending connection context
0809 03:27:45.107620 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 03:27:45.107975 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1597,"thread_start_us":147,"threads_started":1}
I20260809 03:27:45.110474 22893 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34551
I20260809 03:27:45.110731 22893 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:45.111254 22893 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:27:45.112358 22504 ts_manager.cc:194] Registered new tserver with Master: f861eeb34d0e4f8487aaa634d9720316 (127.9.46.194:42943)
I20260809 03:27:45.113150 22504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:57709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:27:45.138733 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:58315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34551
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:45.271047 22899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:45.271481 22899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:45.271615 22899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:45.277268 22899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:45.277406 22899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:45.277521 22899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:27:45.285259 22899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.46.212:58315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:34551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:45.286824 22899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:45.288219 22899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 09 03:27:45 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
W20260809 03:27:45.309298 22905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:45.309414 22907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:45.309759 22904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:45.310667 22899 server_base.cc:1034] running on GCE node
I20260809 03:27:45.311174 22899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:45.311964 22899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:45.319509 22899 hybrid_clock.cc:648] HybridClock initialized: now 1786246065316222 us; error 3287 us; skew 500 ppm
Aug 09 03:27:45 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:45.323333 22899 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:27:45.324491 22899 webserver.cc:460] Webserver started at http://127.9.46.195:43177/ using document root <none> and password file <none>
I20260809 03:27:45.325241 22899 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:45.325341 22899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:45.325592 22899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 03:27:45 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:45.352965 22899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "89cc4c221aac4ab9bb0f6fab4f01d744"
format_stamp: "Formatted at 2026-08-09 03:27:45 on dist-test-slave-65mx"
server_key: "3ef05f758a93b12371ef996dd5a53ce4"
server_key_iv: "70d85dbf323bb9ea56f4d33e84cfe017"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:45.353838 22899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "89cc4c221aac4ab9bb0f6fab4f01d744"
format_stamp: "Formatted at 2026-08-09 03:27:45 on dist-test-slave-65mx"
server_key: "3ef05f758a93b12371ef996dd5a53ce4"
server_key_iv: "70d85dbf323bb9ea56f4d33e84cfe017"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:45.359139 22899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 03:27:45.399359 22915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:45.401209 22899 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.004s sys 0.000s
I20260809 03:27:45.401342 22899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "89cc4c221aac4ab9bb0f6fab4f01d744"
format_stamp: "Formatted at 2026-08-09 03:27:45 on dist-test-slave-65mx"
server_key: "3ef05f758a93b12371ef996dd5a53ce4"
server_key_iv: "70d85dbf323bb9ea56f4d33e84cfe017"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:45.401490 22899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:45.426770 22899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:45.439566 22899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:45.439801 22899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:45.440534 22899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:45.441666 22899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:45.441767 22899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:45.441855 22899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:45.441911 22899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:45.452965 22899 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:42035
I20260809 03:27:45.454217 22899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:27:45.460809 23039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:42035 every 8 connection(s)
I20260809 03:27:45.464416 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 22899
I20260809 03:27:45.464604 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:27:45.464957 9403 external_mini_cluster.cc:1428] Setting key 14da755fa0b99b095bc5b347ff8f16ce
Aug 09 03:27:45 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:45.474622 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:45.460147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:38725 (local address 127.9.46.254:34551)
0809 03:27:45.460271 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:45.460276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:45.460290 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:27:45.460336 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:45.460339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:45.460385 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:45.460468 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:45.462502 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:45.463348 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:45.465045 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:45.465406 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:45.465495 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:45.470315 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:45.470321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:45.470324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:45.470356 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:45.472118 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:45.472787 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:45.472791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:45.472793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:45.472887 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:45.473287 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:45.473290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:45.473292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:45.473510 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:45.473591 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:45.474328 (+ 737us) server_negotiation.cc:299] Negotiation successful
0809 03:27:45.474457 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:27:45.476500 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:45.456834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34551 (local address 127.9.46.195:38725)
0809 03:27:45.457777 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:45.457836 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:45.458875 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:45.461296 (+ 2421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:45.461305 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:45.461671 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:45.462318 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:45.462335 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:45.463510 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:45.463513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:45.464897 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:45.464908 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:45.467224 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:45.467233 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:45.467434 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:45.468150 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:45.468171 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:45.470077 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:45.472245 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:45.472251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:45.472268 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:45.472662 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:45.472992 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:45.472995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:45.472997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:45.473178 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:45.473661 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:45.473667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:45.473971 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 03:27:45.475059 (+ 1088us) client_negotiation.cc:240] Negotiation successful
0809 03:27:45.475462 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":162,"threads_started":1}
I20260809 03:27:45.479372 23040 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34551
I20260809 03:27:45.479604 23040 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:45.480160 23040 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
I20260809 03:27:45.481338 22504 ts_manager.cc:194] Registered new tserver with Master: 89cc4c221aac4ab9bb0f6fab4f01d744 (127.9.46.195:42035)
I20260809 03:27:45.482060 22504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:38725
I20260809 03:27:45.492504 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:27:45.493393 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 22600
I20260809 03:27:45.504961 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 22747
I20260809 03:27:45.514211 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 22899
I20260809 03:27:45.522755 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 22471
I20260809 03:27:45.529544 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34551
--webserver_interface=127.9.46.254
--webserver_port=43009
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:34551
--ranger_config_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:58315/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.46.212:58315/kms with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:45.640226 23043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:45.640484 23043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:45.640524 23043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:45.644490 23043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:27:45.644564 23043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:45.644588 23043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:45.644604 23043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:27:45.644620 23043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:27:45.650022 23043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.46.212:58315/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:34551
--ranger_config_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:34551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=43009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:45.651032 23043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:45.651968 23043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:45.660328 23048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:45.660319 23049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:45.660321 23051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:45.660758 23043 server_base.cc:1034] running on GCE node
I20260809 03:27:45.661230 23043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:45.662271 23043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:45.663466 23043 hybrid_clock.cc:648] HybridClock initialized: now 1786246065663442 us; error 45 us; skew 500 ppm
Aug 09 03:27:45 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:45.666215 23043 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:27:45.667135 23043 webserver.cc:460] Webserver started at http://127.9.46.254:43009/ using document root <none> and password file <none>
I20260809 03:27:45.667613 23043 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:45.667688 23043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:27:45 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:45.706245 23043 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:27:45.708431 23059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:45.709702 23043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:27:45.709836 23043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf"
format_stamp: "Formatted at 2026-08-09 03:27:43 on dist-test-slave-65mx"
server_key: "78233071dc244660443153f21c35689b"
server_key_iv: "d90d05cabcb300ad7ee377a204b35f64"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:45.710373 23043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:45.719231 23043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:45.721771 23043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:45.721967 23043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:45.729549 23043 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:34551
I20260809 03:27:45.729583 23116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:34551 every 8 connection(s)
I20260809 03:27:45.730674 23043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:27:45.735194 23117 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:27:45.737104 23117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: Bootstrap starting.
I20260809 03:27:45.737169 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 23043
I20260809 03:27:45.737709 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:58315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:37317
--local_ip_for_outbound_sockets=127.9.46.193
--tserver_master_addrs=127.9.46.254:34551
--webserver_port=42727
--webserver_interface=127.9.46.193
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.46.212:58315/kms with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:27:45.754952 23117 log.cc:824] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:45.761443 23117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:27:45.761852 23117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: Bootstrap complete.
I20260809 03:27:45.765991 23117 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } }
I20260809 03:27:45.766382 23117 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2775fe1d0a04d25aa2c28aa1d17acaf, State: Initialized, Role: FOLLOWER
I20260809 03:27:45.766836 23117 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } }
I20260809 03:27:45.766973 23117 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:27:45.767067 23117 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:27:45.767187 23117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:27:45.769134 23117 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } }
I20260809 03:27:45.769440 23117 leader_election.cc:302] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2775fe1d0a04d25aa2c28aa1d17acaf; no voters:
I20260809 03:27:45.769737 23117 leader_election.cc:288] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:27:45.769899 23121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:27:45.770144 23121 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [term 2 LEADER]: Becoming Leader. State: Replica: f2775fe1d0a04d25aa2c28aa1d17acaf, State: Running, Role: LEADER
I20260809 03:27:45.770505 23121 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } }
I20260809 03:27:45.770925 23117 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:27:45.771436 23121 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2775fe1d0a04d25aa2c28aa1d17acaf. Latest consensus state: current_term: 2 leader_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } } }
I20260809 03:27:45.771553 23121 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:45.771973 23122 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2775fe1d0a04d25aa2c28aa1d17acaf" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 34551 } } }
I20260809 03:27:45.772112 23122 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:45.772500 23126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:27:45.775710 23126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:27:45.776280 23117 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 03:27:45.776358 23126 catalog_manager.cc:1259] Loaded cluster ID: 2402410c0384455baab97aaea4ac239b
I20260809 03:27:45.776407 23126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:27:45.778720 23126 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:27:45.779386 23126 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: Loaded TSK: 0
I20260809 03:27:45.780578 23126 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:27:45.883262 23119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:45.883590 23119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:45.883669 23119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:45.889992 23119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:45.890158 23119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:45.890305 23119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:27:45.898401 23119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.46.212:58315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:37317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=42727
--tserver_master_addrs=127.9.46.254:34551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:45.900038 23119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:45.901398 23119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:45.914115 23159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:45.914283 23156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:45.914729 23119 server_base.cc:1034] running on GCE node
W20260809 03:27:45.914848 23157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:45.915292 23119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:45.916019 23119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:45.919344 23119 hybrid_clock.cc:648] HybridClock initialized: now 1786246065919311 us; error 37 us; skew 500 ppm
Aug 09 03:27:45 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:45.923038 23119 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:27:45.924180 23119 webserver.cc:460] Webserver started at http://127.9.46.193:42727/ using document root <none> and password file <none>
I20260809 03:27:45.924964 23119 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:45.925113 23119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:27:45 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:45.947261 23119 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:27:45.950377 23169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:45.951932 23119 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:27:45.952073 23119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "5659887875b9424a93ec34c72424b84a"
format_stamp: "Formatted at 2026-08-09 03:27:44 on dist-test-slave-65mx"
server_key: "4ee7fc5e31ba9c729b961d3c5d853f26"
server_key_iv: "0501f565fe970c6d4d0fff6a6b61fd94"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:45.952672 23119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:45.967782 23119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:45.971494 23119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:45.971791 23119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:45.972570 23119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:45.973567 23119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:45.973675 23119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:45.973807 23119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:45.973891 23119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:45.986137 23119 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:37317
I20260809 03:27:45.986302 23292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:37317 every 8 connection(s)
I20260809 03:27:45.987473 23119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:27:45.997987 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 23119
I20260809 03:27:45.998529 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:58315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:42943
--local_ip_for_outbound_sockets=127.9.46.194
--tserver_master_addrs=127.9.46.254:34551
--webserver_port=35357
--webserver_interface=127.9.46.194
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.46.212:58315/kms with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:27:46 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:46.011807 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:45.993677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34551 (local address 127.9.46.193:44111)
0809 03:27:45.994888 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:45.994947 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:45.995984 (+ 1037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:45.998207 (+ 2223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:45.998216 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:45.998562 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:45.999247 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:45.999267 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.001406 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.001409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:46.002250 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:46.002259 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.003029 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.003033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:46.003208 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:46.003848 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:46.003864 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:46.005425 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:46.008091 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:46.008097 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:46.008111 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:46.008456 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:46.008942 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:46.008945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:46.008947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:46.009141 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:46.009932 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:46.009940 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:46.010241 (+ 301us) client_negotiation.cc:769] Sending connection context
0809 03:27:46.010441 (+ 200us) client_negotiation.cc:240] Negotiation successful
0809 03:27:46.010760 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"thread_start_us":157,"threads_started":1}
I20260809 03:27:46.013931 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:45.993569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:44111 (local address 127.9.46.254:34551)
0809 03:27:45.996858 (+ 3289us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:45.996880 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:45.996958 (+ 78us) server_negotiation.cc:407] Connection header received
0809 03:27:45.997267 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:45.997282 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:45.997698 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:45.999493 (+ 1795us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:45.999802 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.001269 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.002379 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.002858 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.003043 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:46.005615 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:46.005621 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:46.005636 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:46.005671 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:46.007923 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:46.008653 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:46.008659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:46.008664 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:46.008818 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:46.009276 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:46.009279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:46.009281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:46.009789 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:46.012519 (+ 2730us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:46.012693 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 03:27:46.012986 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3116,"thread_start_us":116,"threads_started":1}
I20260809 03:27:46.016767 23293 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34551
I20260809 03:27:46.017026 23293 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:46.017617 23293 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
W20260809 03:27:46.019942 23293 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:34551 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 03:27:46.020077 23293 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:46.020252 23293 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
I20260809 03:27:46.021191 23293 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:46.021368 23293 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
W20260809 03:27:46.022149 23293 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:27:46.143725 23297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:46.144107 23297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:46.144218 23297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:46.149161 23297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:46.149246 23297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:46.149376 23297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:27:46.155941 23297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.46.212:58315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:42943
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=35357
--tserver_master_addrs=127.9.46.254:34551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:46.157459 23297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:46.158586 23297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:46.170054 23303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:46.170014 23302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:46.170475 23305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:46.170974 23297 server_base.cc:1034] running on GCE node
I20260809 03:27:46.171458 23297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:46.172243 23297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:46.173441 23297 hybrid_clock.cc:648] HybridClock initialized: now 1786246066173418 us; error 36 us; skew 500 ppm
Aug 09 03:27:46 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246066, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:46.176735 23297 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:27:46.177599 23297 webserver.cc:460] Webserver started at http://127.9.46.194:35357/ using document root <none> and password file <none>
I20260809 03:27:46.178167 23297 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:46.178277 23297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:27:46 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246066, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:46.204064 23297 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.003s
I20260809 03:27:46.207269 23313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:46.208995 23297 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 03:27:46.209131 23297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "f861eeb34d0e4f8487aaa634d9720316"
format_stamp: "Formatted at 2026-08-09 03:27:45 on dist-test-slave-65mx"
server_key: "73604bf283e9a198cb4359e9dbbbe99b"
server_key_iv: "0562200b91dc71c11facf73e17c933c0"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:46.209861 23297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:46.235569 23297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:46.239398 23297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:46.239624 23297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:46.240351 23297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:46.241477 23297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:46.241561 23297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:46.241618 23297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:46.241696 23297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 03:27:46.253557 23297 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:42943
I20260809 03:27:46.261488 23438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:42943 every 8 connection(s)
I20260809 03:27:46.262197 23297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:27:46.267983 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 23297
I20260809 03:27:46.268602 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.46.212:58315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:42035
--local_ip_for_outbound_sockets=127.9.46.195
--tserver_master_addrs=127.9.46.254:34551
--webserver_port=43177
--webserver_interface=127.9.46.195
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.46.212:58315/kms with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:27:46 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246066, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:46.281046 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:46.264214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:48585 (local address 127.9.46.254:34551)
0809 03:27:46.264356 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:46.264361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:46.265523 (+ 1162us) server_negotiation.cc:407] Connection header received
0809 03:27:46.266493 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:46.266496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:46.266566 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:46.266670 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:46.268845 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.269670 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.271668 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.272088 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.272186 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:46.275903 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:46.275909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:46.275914 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:46.275951 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:46.278008 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:46.278738 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:46.278744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:46.278746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:46.278845 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:46.279357 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:46.279363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:46.279408 (+ 45us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:46.279699 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:46.279804 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:46.280632 (+ 828us) server_negotiation.cc:299] Negotiation successful
0809 03:27:46.280798 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:27:46.283269 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:46.264737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34551 (local address 127.9.46.194:48585)
0809 03:27:46.265349 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:46.265412 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:46.266279 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:46.266855 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:46.266861 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:46.267185 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:46.268650 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:46.268671 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.269850 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.269855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:46.271488 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:46.271513 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.272758 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.272764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:46.272999 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:46.273735 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:46.273761 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:46.275674 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:46.278161 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:46.278169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:46.278190 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:46.278598 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:46.278975 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:46.278981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:46.278986 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:46.279206 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:46.279886 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:46.279895 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:46.280254 (+ 359us) client_negotiation.cc:769] Sending connection context
0809 03:27:46.281683 (+ 1429us) client_negotiation.cc:240] Negotiation successful
0809 03:27:46.282079 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":146,"threads_started":1}
I20260809 03:27:46.284610 23439 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34551
I20260809 03:27:46.284853 23439 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:46.285349 23439 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
W20260809 03:27:46.287092 23439 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:34551 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 03:27:46.287237 23439 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:46.287434 23439 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
I20260809 03:27:46.288277 23439 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:46.288460 23439 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
W20260809 03:27:46.289251 23439 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:27:46.407826 23442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:46.408113 23442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:46.408190 23442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:46.412637 23442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:46.412724 23442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:46.412813 23442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:27:46.419106 23442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.46.212:58315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:42035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=43177
--tserver_master_addrs=127.9.46.254:34551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:46.420447 23442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:46.421782 23442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:46.433378 23455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:46.433418 23453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:46.433704 23451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:46.434218 23442 server_base.cc:1034] running on GCE node
I20260809 03:27:46.434635 23442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:46.435338 23442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:46.436525 23442 hybrid_clock.cc:648] HybridClock initialized: now 1786246066436492 us; error 43 us; skew 500 ppm
Aug 09 03:27:46 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246066, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:46.439499 23442 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:27:46.440616 23442 webserver.cc:460] Webserver started at http://127.9.46.195:43177/ using document root <none> and password file <none>
I20260809 03:27:46.441188 23442 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:46.441294 23442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 03:27:46 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246066, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:46.464461 23442 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260809 03:27:46.467790 23463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:46.469249 23442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:27:46.469352 23442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "89cc4c221aac4ab9bb0f6fab4f01d744"
format_stamp: "Formatted at 2026-08-09 03:27:45 on dist-test-slave-65mx"
server_key: "3ef05f758a93b12371ef996dd5a53ce4"
server_key_iv: "70d85dbf323bb9ea56f4d33e84cfe017"
server_key_version: "kuduclusterkey@0"
I20260809 03:27:46.470096 23442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:46.492316 23442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:46.496049 23442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:46.496320 23442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:46.497148 23442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:46.498270 23442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:46.498383 23442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:46.498487 23442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:46.498530 23442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:46.510550 23442 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:42035
I20260809 03:27:46.510579 23585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:42035 every 8 connection(s)
I20260809 03:27:46.511823 23442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:27:46.518548 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 23442
Aug 09 03:27:46 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246066, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:46.529784 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:46.514052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:56009 (local address 127.9.46.254:34551)
0809 03:27:46.514238 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:46.514243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:46.515955 (+ 1712us) server_negotiation.cc:407] Connection header received
0809 03:27:46.517123 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:46.517127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:46.517191 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:46.517269 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:46.518527 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.519364 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.520914 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.521275 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.521361 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:46.525286 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:46.525290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:46.525293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:46.525320 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:46.527028 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:46.527668 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:46.527671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:46.527673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:46.527760 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:46.528254 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:46.528258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:46.528260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:46.528473 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:46.528603 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:46.529494 (+ 891us) server_negotiation.cc:299] Negotiation successful
0809 03:27:46.529614 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 03:27:46.531533 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:46.514527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:34551 (local address 127.9.46.195:56009)
0809 03:27:46.515772 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:46.515832 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:46.516890 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:46.517445 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:46.517451 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:46.517727 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:46.518337 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:46.518359 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.519557 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.519563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:46.520760 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:46.520771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.521574 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.521578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:46.521776 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:46.522620 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:46.522642 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:46.525071 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:46.527168 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:46.527174 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:46.527191 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:46.527545 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:46.527864 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:46.527867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:46.527869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:46.528129 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:46.528663 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:46.528670 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:46.528996 (+ 326us) client_negotiation.cc:769] Sending connection context
0809 03:27:46.529200 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 03:27:46.530464 (+ 1264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"thread_start_us":158,"threads_started":1}
I20260809 03:27:46.533020 23586 heartbeater.cc:347] Connected to a master server at 127.9.46.254:34551
I20260809 03:27:46.533250 23586 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:46.533875 23586 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
W20260809 03:27:46.535540 23586 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:34551 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 03:27:46.535686 23586 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:46.535856 23586 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
I20260809 03:27:46.537209 23586 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:46.537382 23586 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
W20260809 03:27:46.538177 23586 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 03:27:46.543406 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:46.519699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54946 (local address 127.9.46.254:34551)
0809 03:27:46.523555 (+ 3856us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:46.523567 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:46.523581 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:27:46.523617 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:46.523620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:46.523666 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:46.525612 (+ 1946us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:46.525737 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.533231 (+ 7494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.534504 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:46.536239 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:46.539781 (+ 3542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:46.539904 (+ 123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:46.539908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:46.539911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:46.539929 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:46.541687 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:46.542230 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:46.542233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:46.542234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:46.542314 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:46.542622 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:46.542624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:46.542625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:46.542791 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:46.542934 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:46.543151 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 03:27:46.543249 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3752,"thread_start_us":98,"threads_started":1}
Aug 09 03:27:46 dist-test-slave-65mx krb5kdc[21982](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786246066, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:47.022377 23293 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:47.022732 23293 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
I20260809 03:27:47.024044 23078 ts_manager.cc:194] Registered new tserver with Master: 5659887875b9424a93ec34c72424b84a (127.9.46.193:37317)
I20260809 03:27:47.025808 23078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:44111
I20260809 03:27:47.289500 23439 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:47.289798 23439 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
I20260809 03:27:47.292467 23078 ts_manager.cc:194] Registered new tserver with Master: f861eeb34d0e4f8487aaa634d9720316 (127.9.46.194:42943)
I20260809 03:27:47.299031 23078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:48585
Aug 09 03:27:47 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246066, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:47.538470 23586 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:47.538831 23586 heartbeater.cc:511] Master 127.9.46.254:34551 requested a full tablet report, sending...
I20260809 03:27:47.539834 23078 ts_manager.cc:194] Registered new tserver with Master: 89cc4c221aac4ab9bb0f6fab4f01d744 (127.9.46.195:42035)
I20260809 03:27:47.540688 23078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:56009
I20260809 03:27:47.541116 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 03:27:47 dist-test-slave-65mx krb5kdc[21982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246066, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for HTTP/127.9.46.212@KRBTEST.COM
I20260809 03:27:47.562759 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.545675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.9.46.254:34551)
0809 03:27:47.545952 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:47.545957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:47.551395 (+ 5438us) server_negotiation.cc:407] Connection header received
0809 03:27:47.551455 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:47.551458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:47.551515 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:47.551615 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:47.552709 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.553555 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.554795 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.555158 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.555262 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.557665 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:47.557669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:47.557672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:47.557707 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:47.560389 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:47.561289 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:47.561292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:47.561294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:47.561400 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:47.561731 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:47.561734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:47.561736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:47.561987 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:47.562129 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:47.562435 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 03:27:47.562576 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":105,"threads_started":1}
I20260809 03:27:47.567492 23078 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:27:47.573053 23078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:27:47.605628 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.593747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44014 (local address 127.9.46.195:42035)
0809 03:27:47.596813 (+ 3066us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:47.596820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:47.597279 (+ 459us) server_negotiation.cc:407] Connection header received
0809 03:27:47.597368 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:47.597373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:47.598375 (+ 1002us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:47.598536 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:47.599038 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.600266 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.602945 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.603865 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.603982 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.604134 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:47.605355 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0809 03:27:47.605463 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2959,"thread_start_us":137,"threads_started":1}
I20260809 03:27:47.606163 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.593504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:42035 (local address 127.0.0.1:44014)
0809 03:27:47.596906 (+ 3402us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:47.596978 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:47.597123 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:47.598707 (+ 1584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:47.598713 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:47.598736 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:47.598898 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:47.598906 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.600402 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.600406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:47.602795 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:47.602806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.604216 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.604219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:47.604335 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.604376 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:27:47.604492 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 03:27:47.606013 (+ 1521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3205,"mutex_wait_us":75,"thread_start_us":128,"threads_started":1}
I20260809 03:27:47.608378 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.595874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.9.46.193:37317)
0809 03:27:47.596179 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:47.596186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:47.597104 (+ 918us) server_negotiation.cc:407] Connection header received
0809 03:27:47.598280 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:47.598286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:47.598463 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:47.598574 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:47.601969 (+ 3395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.603236 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.606756 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.607552 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.607638 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.607764 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:47.608131 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 03:27:47.608236 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":133,"threads_started":1}
I20260809 03:27:47.609009 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.594947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:37317 (local address 127.0.0.1:32946)
0809 03:27:47.596985 (+ 2038us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:47.597000 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:47.598098 (+ 1098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:47.598746 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:47.598749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:47.598767 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:47.601804 (+ 3037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:47.601816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.603399 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.603402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:47.604624 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:47.604635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.607869 (+ 3234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.607873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:47.607967 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.607990 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:27:47.608779 (+ 789us) client_negotiation.cc:240] Negotiation successful
0809 03:27:47.608865 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1953,"mutex_wait_us":2901,"thread_start_us":105,"threads_started":1}
I20260809 03:27:47.610013 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.595537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.9.46.194:42943)
0809 03:27:47.595875 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:47.595883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:47.597491 (+ 1608us) server_negotiation.cc:407] Connection header received
0809 03:27:47.597573 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:47.597579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:47.597779 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:47.597921 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:47.599104 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.600316 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.605237 (+ 4921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.606397 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.606523 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.606664 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:47.609758 (+ 3094us) server_negotiation.cc:299] Negotiation successful
0809 03:27:47.609867 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":102,"threads_started":1}
I20260809 03:27:47.609710 23508 tablet_service.cc:1467] Processing CreateTablet for tablet 661ba24ba9ba473b92ad6c7e6133bc87 (DEFAULT_TABLE table=test-table [id=41f9eb416510423dae39b9c24efc2dc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:27:47.610921 23508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 661ba24ba9ba473b92ad6c7e6133bc87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:47.613397 23215 tablet_service.cc:1467] Processing CreateTablet for tablet 661ba24ba9ba473b92ad6c7e6133bc87 (DEFAULT_TABLE table=test-table [id=41f9eb416510423dae39b9c24efc2dc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:27:47.614511 23215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 661ba24ba9ba473b92ad6c7e6133bc87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:47.624929 23625 tablet_bootstrap.cc:492] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744: Bootstrap starting.
I20260809 03:27:47.609655 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.595332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:42943 (local address 127.0.0.1:55066)
0809 03:27:47.597258 (+ 1926us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:47.597270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:47.597364 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:47.598851 (+ 1487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:47.598854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:47.598864 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:47.598946 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:47.598969 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.600627 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.600630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:47.605082 (+ 4452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:47.605093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.609253 (+ 4160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.609257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:47.609350 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.609374 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:27:47.609449 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 03:27:47.609527 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1868,"mutex_wait_us":2993,"thread_start_us":93,"threads_started":1}
I20260809 03:27:47.628161 23626 tablet_bootstrap.cc:492] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a: Bootstrap starting.
I20260809 03:27:47.629769 23625 tablet_bootstrap.cc:654] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:47.630733 23625 log.cc:824] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:47.630828 23626 tablet_bootstrap.cc:654] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:47.631742 23626 log.cc:824] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:47.634249 23625 tablet_bootstrap.cc:492] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744: No bootstrap required, opened a new log
I20260809 03:27:47.634474 23625 ts_tablet_manager.cc:1397] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260809 03:27:47.636961 23348 tablet_service.cc:1467] Processing CreateTablet for tablet 661ba24ba9ba473b92ad6c7e6133bc87 (DEFAULT_TABLE table=test-table [id=41f9eb416510423dae39b9c24efc2dc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:27:47.638120 23348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 661ba24ba9ba473b92ad6c7e6133bc87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:47.638409 23625 raft_consensus.cc:348] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 } } peers { permanent_uuid: "5659887875b9424a93ec34c72424b84a" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37317 } } peers { permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 } }
I20260809 03:27:47.638700 23625 raft_consensus.cc:374] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:47.638758 23625 raft_consensus.cc:729] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89cc4c221aac4ab9bb0f6fab4f01d744, State: Initialized, Role: FOLLOWER
I20260809 03:27:47.639179 23625 consensus_queue.cc:260] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 } } peers { permanent_uuid: "5659887875b9424a93ec34c72424b84a" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37317 } } peers { permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 } }
I20260809 03:27:47.639801 23586 heartbeater.cc:503] Master 127.9.46.254:34551 was elected leader, sending a full tablet report...
I20260809 03:27:47.640204 23625 ts_tablet_manager.cc:1428] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:27:47.648617 23630 tablet_bootstrap.cc:492] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316: Bootstrap starting.
I20260809 03:27:47.651456 23626 tablet_bootstrap.cc:492] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a: No bootstrap required, opened a new log
I20260809 03:27:47.651687 23626 ts_tablet_manager.cc:1397] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.000s
I20260809 03:27:47.654604 23630 tablet_bootstrap.cc:654] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:47.655586 23630 log.cc:824] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:47.656298 23626 raft_consensus.cc:348] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 } } peers { permanent_uuid: "5659887875b9424a93ec34c72424b84a" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37317 } } peers { permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 } }
I20260809 03:27:47.656665 23626 raft_consensus.cc:374] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:47.656766 23626 raft_consensus.cc:729] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5659887875b9424a93ec34c72424b84a, State: Initialized, Role: FOLLOWER
I20260809 03:27:47.657330 23626 consensus_queue.cc:260] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 } } peers { permanent_uuid: "5659887875b9424a93ec34c72424b84a" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37317 } } peers { permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 } }
I20260809 03:27:47.660048 23626 ts_tablet_manager.cc:1428] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 03:27:47.660431 23630 tablet_bootstrap.cc:492] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316: No bootstrap required, opened a new log
I20260809 03:27:47.660493 23293 heartbeater.cc:503] Master 127.9.46.254:34551 was elected leader, sending a full tablet report...
I20260809 03:27:47.660674 23630 ts_tablet_manager.cc:1397] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260809 03:27:47.664637 23630 raft_consensus.cc:348] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 } } peers { permanent_uuid: "5659887875b9424a93ec34c72424b84a" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37317 } } peers { permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 } }
I20260809 03:27:47.664963 23630 raft_consensus.cc:374] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:47.665073 23630 raft_consensus.cc:729] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f861eeb34d0e4f8487aaa634d9720316, State: Initialized, Role: FOLLOWER
I20260809 03:27:47.665737 23630 consensus_queue.cc:260] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 } } peers { permanent_uuid: "5659887875b9424a93ec34c72424b84a" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37317 } } peers { permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 } }
I20260809 03:27:47.666913 23630 ts_tablet_manager.cc:1428] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:27:47.667302 23439 heartbeater.cc:503] Master 127.9.46.254:34551 was elected leader, sending a full tablet report...
I20260809 03:27:47.669456 23632 raft_consensus.cc:482] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:27:47.669629 23632 raft_consensus.cc:504] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 } } peers { permanent_uuid: "5659887875b9424a93ec34c72424b84a" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37317 } } peers { permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 } }
I20260809 03:27:47.670928 23632 leader_election.cc:288] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89cc4c221aac4ab9bb0f6fab4f01d744 (127.9.46.195:42035), f861eeb34d0e4f8487aaa634d9720316 (127.9.46.194:42943)
I20260809 03:27:47.679584 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.672136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:46229 (local address 127.9.46.195:42035)
0809 03:27:47.672272 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:47.672276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:47.675261 (+ 2985us) server_negotiation.cc:407] Connection header received
0809 03:27:47.675332 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:47.675335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:47.675390 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:47.675490 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:47.675776 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.676535 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.678165 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.678801 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.678911 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.678983 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:47.679361 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 03:27:47.679444 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:27:47.680204 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.672005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:42035 (local address 127.9.46.193:46229)
0809 03:27:47.675149 (+ 3144us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:47.675162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:47.675252 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:47.675506 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:47.675508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:47.675525 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:47.675653 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:47.675659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.676770 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.676774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:47.678011 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:47.678023 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.679095 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.679099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:47.679208 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.679233 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 03:27:47.679942 (+ 709us) client_negotiation.cc:240] Negotiation successful
0809 03:27:47.680037 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3064,"mutex_wait_us":49,"thread_start_us":93,"threads_started":1}
I20260809 03:27:47.681075 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "661ba24ba9ba473b92ad6c7e6133bc87" candidate_uuid: "5659887875b9424a93ec34c72424b84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" is_pre_election: true
I20260809 03:27:47.681416 23526 raft_consensus.cc:2454] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5659887875b9424a93ec34c72424b84a in term 0.
I20260809 03:27:47.681942 23174 leader_election.cc:302] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5659887875b9424a93ec34c72424b84a, 89cc4c221aac4ab9bb0f6fab4f01d744; no voters:
I20260809 03:27:47.682250 23632 raft_consensus.cc:2781] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:27:47.682353 23632 raft_consensus.cc:482] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:27:47.682399 23632 raft_consensus.cc:3037] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:47.683770 23632 raft_consensus.cc:504] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 } } peers { permanent_uuid: "5659887875b9424a93ec34c72424b84a" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37317 } } peers { permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 } }
I20260809 03:27:47.684175 23632 leader_election.cc:288] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [CANDIDATE]: Term 1 election: Requested vote from peers 89cc4c221aac4ab9bb0f6fab4f01d744 (127.9.46.195:42035), f861eeb34d0e4f8487aaa634d9720316 (127.9.46.194:42943)
I20260809 03:27:47.684517 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "661ba24ba9ba473b92ad6c7e6133bc87" candidate_uuid: "5659887875b9424a93ec34c72424b84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744"
I20260809 03:27:47.684680 23526 raft_consensus.cc:3037] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:47.686033 23526 raft_consensus.cc:2454] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5659887875b9424a93ec34c72424b84a in term 1.
I20260809 03:27:47.686335 23174 leader_election.cc:302] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5659887875b9424a93ec34c72424b84a, 89cc4c221aac4ab9bb0f6fab4f01d744; no voters:
I20260809 03:27:47.686563 23632 raft_consensus.cc:2781] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:27:47.686786 23632 raft_consensus.cc:686] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [term 1 LEADER]: Becoming Leader. State: Replica: 5659887875b9424a93ec34c72424b84a, State: Running, Role: LEADER
I20260809 03:27:47.687175 23632 consensus_queue.cc:237] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 } } peers { permanent_uuid: "5659887875b9424a93ec34c72424b84a" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37317 } } peers { permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 } }
I20260809 03:27:47.690052 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.671660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:47113 (local address 127.9.46.194:42943)
0809 03:27:47.671772 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:47.671776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:47.674812 (+ 3036us) server_negotiation.cc:407] Connection header received
0809 03:27:47.675009 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:47.675012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:47.675080 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:47.675148 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:47.675619 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.676399 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.688673 (+ 12274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.689300 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.689408 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.689471 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:47.689837 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 03:27:47.689925 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 03:27:47.690649 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.671583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:42943 (local address 127.9.46.193:47113)
0809 03:27:47.674703 (+ 3120us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:47.674717 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:47.674884 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:47.675341 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:47.675343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:47.675353 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:47.675494 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:47.675500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.676570 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.676574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:47.678114 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:47.678126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.689576 (+ 11450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.689579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:47.689682 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.689705 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:27:47.690379 (+ 674us) client_negotiation.cc:240] Negotiation successful
0809 03:27:47.690478 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3057,"mutex_wait_us":136,"thread_start_us":116,"threads_started":1}
I20260809 03:27:47.690647 23078 catalog_manager.cc:5495] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5659887875b9424a93ec34c72424b84a (127.9.46.193). New cstate: current_term: 1 leader_uuid: "5659887875b9424a93ec34c72424b84a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5659887875b9424a93ec34c72424b84a" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 37317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 } health_report { overall_health: UNKNOWN } } }
I20260809 03:27:47.691429 23386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "661ba24ba9ba473b92ad6c7e6133bc87" candidate_uuid: "5659887875b9424a93ec34c72424b84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f861eeb34d0e4f8487aaa634d9720316" is_pre_election: true
I20260809 03:27:47.691736 23386 raft_consensus.cc:2454] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5659887875b9424a93ec34c72424b84a in term 0.
I20260809 03:27:47.692075 23385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "661ba24ba9ba473b92ad6c7e6133bc87" candidate_uuid: "5659887875b9424a93ec34c72424b84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f861eeb34d0e4f8487aaa634d9720316"
I20260809 03:27:47.692185 23385 raft_consensus.cc:3037] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:47.693576 23385 raft_consensus.cc:2454] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5659887875b9424a93ec34c72424b84a in term 1.
I20260809 03:27:47.738745 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.733416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.9.46.193:37317)
0809 03:27:47.733556 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:47.733560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:47.733599 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:27:47.733791 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:47.733794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:47.733871 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:47.734004 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:27:47.734441 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.735232 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.736440 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:47.736808 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:47.736915 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:47.737373 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:27:47.737948 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:27:47.738069 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:47.738515 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 03:27:47.738582 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 03:27:47.739708 23294 tablet.cc:2369] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:27:47.748869 23385 raft_consensus.cc:1261] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316 [term 1 FOLLOWER]: Refusing update from remote peer 5659887875b9424a93ec34c72424b84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:27:47.748987 23526 raft_consensus.cc:1261] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744 [term 1 FOLLOWER]: Refusing update from remote peer 5659887875b9424a93ec34c72424b84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:27:47.749656 23636 consensus_queue.cc:1035] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "89cc4c221aac4ab9bb0f6fab4f01d744" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:27:47.749845 23632 consensus_queue.cc:1035] T 661ba24ba9ba473b92ad6c7e6133bc87 P 5659887875b9424a93ec34c72424b84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f861eeb34d0e4f8487aaa634d9720316" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 42943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:27:47.759532 23640 mvcc.cc:205] Tried to move back new op lower bound from 7316463893489786880 to 7316463893250236416. Current Snapshot: MvccSnapshot[applied={T|T < 7316463893489786880}]
W20260809 03:27:47.764006 23587 tablet.cc:2369] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:27:47.764147 23440 tablet.cc:2369] T 661ba24ba9ba473b92ad6c7e6133bc87 P f861eeb34d0e4f8487aaa634d9720316: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:27:47.767318 23641 mvcc.cc:205] Tried to move back new op lower bound from 7316463893489786880 to 7316463893250236416. Current Snapshot: MvccSnapshot[applied={T|T < 7316463893489786880}]
I20260809 03:27:47.769506 23642 mvcc.cc:205] Tried to move back new op lower bound from 7316463893489786880 to 7316463893250236416. Current Snapshot: MvccSnapshot[applied={T|T < 7316463893489786880}]
I20260809 03:27:47.779650 23078 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:27:47.779836 23078 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:27:47.782735 23078 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2775fe1d0a04d25aa2c28aa1d17acaf: Sending DeleteTablet for 3 replicas of tablet 661ba24ba9ba473b92ad6c7e6133bc87
I20260809 03:27:47.783926 23215 tablet_service.cc:1514] Processing DeleteTablet for tablet 661ba24ba9ba473b92ad6c7e6133bc87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:27:47 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:32946
I20260809 03:27:47.784444 23348 tablet_service.cc:1514] Processing DeleteTablet for tablet 661ba24ba9ba473b92ad6c7e6133bc87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:27:47 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:55066
I20260809 03:27:47.785642 23508 tablet_service.cc:1514] Processing DeleteTablet for tablet 661ba24ba9ba473b92ad6c7e6133bc87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:27:47 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:44014
I20260809 03:27:47.786765 23652 tablet_replica.cc:330] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744: stopping tablet replica
I20260809 03:27:47.787140 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 23119
W20260809 03:27:47.799518 23061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:37317 (error 108)
I20260809 03:27:47.800624 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 23297
I20260809 03:27:47.804926 23652 raft_consensus.cc:2229] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:27:47.805483 23652 raft_consensus.cc:2258] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:27:47.812424 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.812072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:37317 (local address 127.0.0.1:32964)
0809 03:27:47.812242 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:47.812326 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:37317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 03:27:47.812803 23061 catalog_manager.cc:4584] TS 5659887875b9424a93ec34c72424b84a (127.9.46.193:37317): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 661ba24ba9ba473b92ad6c7e6133bc87: Network error: Client connection negotiation failed: client connection to 127.9.46.193:37317: connect: Connection refused (error 111)
I20260809 03:27:47.815219 23652 ts_tablet_manager.cc:1905] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 03:27:47.816824 23621 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:27:47.813421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:42943 (local address 127.0.0.1:55082)
0809 03:27:47.813594 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:47.813609 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:47.813704 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:47.816719 (+ 3015us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.194:42943: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 03:27:47.819115 23652 ts_tablet_manager.cc:1918] T 661ba24ba9ba473b92ad6c7e6133bc87 P 89cc4c221aac4ab9bb0f6fab4f01d744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:27:47.819440 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 23442
I20260809 03:27:47.828729 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:47.828418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:42035 (local address 127.0.0.1:44024)
0809 03:27:47.828580 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:47.828641 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.195:42035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 03:27:47.828871 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 23043
I20260809 03:27:47.837103 9403 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 03:27:48.182212 9403 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 03:27:48.182358 9403 mini_ranger.cc:66] Stopping Ranger...
I20260809 03:27:48.527283 9403 mini_ranger.cc:68] Stopped Ranger
2026-08-09 03:27:48.527 UTC [22015] LOG: received smart shutdown request
2026-08-09 03:27:48.531 UTC [22015] LOG: background worker "logical replication launcher" (PID 22025) exited with exit code 1
2026-08-09 03:27:48.532 UTC [22019] LOG: shutting down
2026-08-09 03:27:48.702 UTC [22015] LOG: database system is shut down
2026-08-09T03:27:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49460 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:27:48 dist-test-slave-65mx krb5kdc[23664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:27:48 dist-test-slave-65mx krb5kdc[23664](info): set up 2 sockets
Aug 09 03:27:48 dist-test-slave-65mx krb5kdc[23664](info): commencing operation
krb5kdc: starting...
W20260809 03:27:49.917423 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:27:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:27:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:27:50.048324 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41715
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:44923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:41715,127.9.46.253:37469,127.9.46.252:36045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:50.159659 23677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:50.159981 23677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:50.160081 23677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:50.164354 23677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:27:50.164443 23677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:50.164479 23677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:50.164522 23677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:27:50.164597 23677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:27:50.170006 23677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:41715,127.9.46.253:37469,127.9.46.252:36045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:50.171139 23677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:50.172165 23677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:50.180613 23686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:50.180677 23682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:50.180748 23677 server_base.cc:1034] running on GCE node
W20260809 03:27:50.180677 23683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:50.181470 23677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:50.182530 23677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:50.183701 23677 hybrid_clock.cc:648] HybridClock initialized: now 1786246070183685 us; error 25 us; skew 500 ppm
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:50.186636 23677 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:27:50.187516 23677 webserver.cc:460] Webserver started at http://127.9.46.254:43117/ using document root <none> and password file <none>
I20260809 03:27:50.188024 23677 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:50.188082 23677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:50.188243 23677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:27:50.190279 23677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "124a20296b374926979568512deb5804"
format_stamp: "Formatted at 2026-08-09 03:27:50 on dist-test-slave-65mx"
server_key: "4bfe85987c9f4f9b94dacdd676197c0c"
server_key_iv: "dada80b3df7ef69a160ad32e6fd4e107"
server_key_version: "encryptionkey@0"
I20260809 03:27:50.190819 23677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "124a20296b374926979568512deb5804"
format_stamp: "Formatted at 2026-08-09 03:27:50 on dist-test-slave-65mx"
server_key: "4bfe85987c9f4f9b94dacdd676197c0c"
server_key_iv: "dada80b3df7ef69a160ad32e6fd4e107"
server_key_version: "encryptionkey@0"
I20260809 03:27:50.194654 23677 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 03:27:50.197136 23693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:50.198339 23677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:27:50.198478 23677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "124a20296b374926979568512deb5804"
format_stamp: "Formatted at 2026-08-09 03:27:50 on dist-test-slave-65mx"
server_key: "4bfe85987c9f4f9b94dacdd676197c0c"
server_key_iv: "dada80b3df7ef69a160ad32e6fd4e107"
server_key_version: "encryptionkey@0"
I20260809 03:27:50.198589 23677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:50.214684 23677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:50.217705 23677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:50.217917 23677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:50.225106 23677 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:41715
I20260809 03:27:50.225135 23753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:41715 every 8 connection(s)
I20260809 03:27:50.226047 23677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:27:50.228716 23754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:50.232304 23754 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:27:50.235188 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 23677
I20260809 03:27:50.235327 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:27:50.235610 9403 external_mini_cluster.cc:1428] Setting key 61d4afb256b565b1bef0e7fc5c335626
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.46.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.253@KRBTEST.COM" created.
I20260809 03:27:50.247911 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.234539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41715 (local address 127.0.0.1:45782)
0809 03:27:50.234995 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.235044 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:50.235948 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:50.236746 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:50.236750 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:50.237093 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:50.237339 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.237351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.238599 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.238604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.239477 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.239487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.240626 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.240632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.240773 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.241386 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:50.241402 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:50.242865 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:50.245237 (+ 2372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.245246 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.245253 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.245471 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.245766 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.245768 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.245769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.245873 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.246508 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:50.246513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:50.246647 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 03:27:50.246788 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 03:27:50.247066 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":1,"spinlock_wait_cycles":33664,"thread_start_us":151,"threads_started":1}
I20260809 03:27:50.247910 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.234539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45782 (local address 127.9.46.254:41715)
0809 03:27:50.234943 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.234964 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.235160 (+ 196us) server_negotiation.cc:407] Connection header received
0809 03:27:50.236248 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.236256 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.236489 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.236722 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.237492 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.238446 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.239700 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.240379 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.240591 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.243031 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.243039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.243052 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.243082 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.245131 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.245575 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.245579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.245580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.245685 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.245956 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.245958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.245959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.246406 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.246513 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.246858 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.247157 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":266,"thread_start_us":156,"threads_started":1}
I20260809 03:27:50.250288 23754 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:27:50.250975 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.250741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:37469 (local address 127.0.0.1:48200)
0809 03:27:50.250851 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.250924 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 03:27:50.251221 23697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.46.253:37469: connect: Connection refused (error 111)
I20260809 03:27:50.252092 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.251901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:37469 (local address 127.0.0.1:48210)
0809 03:27:50.251997 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.252039 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 03:27:50.252307 23754 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.253:37469: Network error: Client connection negotiation failed: client connection to 127.9.46.253:37469: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab.
Entry for principal kudu/127.9.46.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab.
I20260809 03:27:50.263692 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.253:37469
--webserver_interface=127.9.46.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab
--principal=kudu/127.9.46.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:44923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:41715,127.9.46.253:37469,127.9.46.252:36045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:27:50.301595 23754 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } attempt: 1
I20260809 03:27:50.302539 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.302226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:37469 (local address 127.0.0.1:48224)
0809 03:27:50.302368 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.302436 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 03:27:50.303320 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.303109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:37469 (local address 127.0.0.1:48234)
0809 03:27:50.303216 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.303264 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 03:27:50.303474 23754 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.253:37469: Network error: Client connection negotiation failed: client connection to 127.9.46.253:37469: connect: Connection refused (error 111)
I20260809 03:27:50.371702 23754 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } attempt: 2
I20260809 03:27:50.372660 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.372300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:37469 (local address 127.0.0.1:48250)
0809 03:27:50.372471 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.372566 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 03:27:50.373442 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.373232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:37469 (local address 127.0.0.1:48266)
0809 03:27:50.373336 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.373386 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:37469: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 03:27:50.373588 23754 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.253:37469: Network error: Client connection negotiation failed: client connection to 127.9.46.253:37469: connect: Connection refused (error 111)
W20260809 03:27:50.377031 23761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:50.377280 23761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:50.377354 23761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:50.381664 23761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:27:50.381726 23761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:50.381747 23761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:50.381780 23761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:27:50.381816 23761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:27:50.387180 23761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:41715,127.9.46.253:37469,127.9.46.252:36045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab
--principal=kudu/127.9.46.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.253:37469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.46.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:50.388271 23761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:50.389346 23761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:27:50.397825 23761 server_base.cc:1034] running on GCE node
W20260809 03:27:50.397782 23768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:50.397782 23767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:50.397780 23770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:50.398641 23761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:50.399744 23761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:50.400898 23761 hybrid_clock.cc:648] HybridClock initialized: now 1786246070400879 us; error 31 us; skew 500 ppm
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:50.403828 23761 init.cc:377] Logged in from keytab as kudu/127.9.46.253@KRBTEST.COM (short username kudu)
I20260809 03:27:50.404772 23761 webserver.cc:460] Webserver started at http://127.9.46.253:37371/ using document root <none> and password file <none>
I20260809 03:27:50.405287 23761 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:50.405345 23761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:50.405510 23761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:27:50.407593 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/data/instance:
uuid: "ec92591f7a814b01b8432a1b82ce4d1e"
format_stamp: "Formatted at 2026-08-09 03:27:50 on dist-test-slave-65mx"
server_key: "f9469e207b8a62188a56c9f0a440a8f3"
server_key_iv: "9372124d51eb3fa7da1d7cf9fbd4934c"
server_key_version: "encryptionkey@0"
I20260809 03:27:50.408128 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/wal/instance:
uuid: "ec92591f7a814b01b8432a1b82ce4d1e"
format_stamp: "Formatted at 2026-08-09 03:27:50 on dist-test-slave-65mx"
server_key: "f9469e207b8a62188a56c9f0a440a8f3"
server_key_iv: "9372124d51eb3fa7da1d7cf9fbd4934c"
server_key_version: "encryptionkey@0"
I20260809 03:27:50.411815 23761 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 03:27:50.414129 23777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:50.415288 23761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:27:50.415426 23761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/wal
uuid: "ec92591f7a814b01b8432a1b82ce4d1e"
format_stamp: "Formatted at 2026-08-09 03:27:50 on dist-test-slave-65mx"
server_key: "f9469e207b8a62188a56c9f0a440a8f3"
server_key_iv: "9372124d51eb3fa7da1d7cf9fbd4934c"
server_key_version: "encryptionkey@0"
I20260809 03:27:50.415534 23761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:50.426847 23761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:50.429873 23761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:50.430107 23761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:50.437502 23761 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.253:37469
I20260809 03:27:50.437526 23836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.253:37469 every 8 connection(s)
I20260809 03:27:50.438474 23761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
I20260809 03:27:50.440455 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 23761
I20260809 03:27:50.440605 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-1/wal/instance
I20260809 03:27:50.440908 9403 external_mini_cluster.cc:1428] Setting key d36cb40a51a04832a07ce3da8e6a82d9
I20260809 03:27:50.441366 23837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:50.445163 23837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.46.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.252@KRBTEST.COM" created.
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:50.459215 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.447043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45794 (local address 127.9.46.254:41715)
0809 03:27:50.447230 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.447238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.448030 (+ 792us) server_negotiation.cc:407] Connection header received
0809 03:27:50.449028 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.449032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.449105 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.449228 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.450415 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.450942 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.452283 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.452735 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.452869 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.455556 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.455560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.455562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.455585 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.457079 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.457603 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.457608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.457610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.457693 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.458055 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.458060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.458061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.458255 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.458361 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.458893 (+ 532us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.459028 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 03:27:50.459992 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.447372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41715 (local address 127.0.0.1:45794)
0809 03:27:50.447857 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.447909 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:50.448822 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:50.449381 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:50.449388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:50.449704 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:50.450281 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.450297 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.451157 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.451161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.452158 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.452168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.452883 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.452886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.453025 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.453697 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:50.453723 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:50.455380 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:50.457206 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.457212 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.457228 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.457493 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.457784 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.457788 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.457790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.457937 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.458361 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:50.458369 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:50.458634 (+ 265us) client_negotiation.cc:769] Sending connection context
0809 03:27:50.458822 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 03:27:50.459151 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":144,"threads_started":1}
I20260809 03:27:50.461453 23837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:27:50.464773 23754 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab.
Entry for principal kudu/127.9.46.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab.
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
I20260809 03:27:50.468144 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.252:36045
--webserver_interface=127.9.46.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab
--principal=kudu/127.9.46.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:44923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:41715,127.9.46.253:37469,127.9.46.252:36045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
I20260809 03:27:50.472886 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.461975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:37469 (local address 127.0.0.1:48268)
0809 03:27:50.462089 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.462103 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:50.462177 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:50.462722 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:50.462725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:50.462898 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:50.462979 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.462984 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.463975 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.463978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.464652 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.464662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.465410 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.465417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.465521 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.466075 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:50.466089 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:50.467666 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:50.470699 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.470701 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.470703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.470881 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.471199 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.471202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.471205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.471316 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.472137 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:50.472143 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:50.472257 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 03:27:50.472379 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 03:27:50.472623 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 03:27:50.473110 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.462089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.9.46.253:37469)
0809 03:27:50.462410 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.462418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.462436 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:27:50.462466 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.462470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.462610 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.462745 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.463082 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.463849 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.464764 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.465273 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.465380 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.467835 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.467847 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.467862 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.467890 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.470558 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.470995 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.471001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.471003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.471088 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.471425 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.471428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.471429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.471767 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.471923 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.472497 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.472742 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":242,"thread_start_us":144,"threads_started":1}
I20260809 03:27:50.475420 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.465186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:37469 (local address 127.0.0.1:48284)
0809 03:27:50.465368 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.465383 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:50.465453 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:50.465860 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:50.465862 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:50.466046 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:50.466756 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.466767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.467640 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.467644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.468289 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.468304 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.468882 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.468885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.468994 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.469420 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:50.469434 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:50.470965 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:50.472879 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.472882 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.472884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.473126 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.473569 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.473572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.473574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.473663 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.474016 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:50.474019 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:50.474128 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 03:27:50.475122 (+ 994us) client_negotiation.cc:240] Negotiation successful
0809 03:27:50.475274 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 03:27:50.476497 23837 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:27:50.477213 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.465260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48284 (local address 127.9.46.253:37469)
0809 03:27:50.465662 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.465669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.465678 (+ 9us) server_negotiation.cc:407] Connection header received
0809 03:27:50.465703 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.465705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.465735 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.465839 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.466872 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.467491 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.468433 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.468763 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.468878 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.471083 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.471086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.471088 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.471108 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.472756 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.473378 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.473384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.473386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.473467 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.473759 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.473762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.473764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.473924 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.474022 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.475198 (+ 1176us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.477008 (+ 1810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":147,"threads_started":1}
I20260809 03:27:50.477921 23754 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:27:50.478551 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.478345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:37968)
0809 03:27:50.478447 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.478496 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 03:27:50.479027 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.478339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:37962)
0809 03:27:50.478875 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.478956 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464}
W20260809 03:27:50.479321 23779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
I20260809 03:27:50.480228 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.480037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38000)
0809 03:27:50.480142 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.480184 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 03:27:50.480412 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.479039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:37984)
0809 03:27:50.480264 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.480328 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1136}
W20260809 03:27:50.480607 23754 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:36045: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
W20260809 03:27:50.480669 23837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:36045: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
I20260809 03:27:50.511782 23754 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } attempt: 1
I20260809 03:27:50.512713 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.512342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38008)
0809 03:27:50.512518 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.512620 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 03:27:50.513597 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.513304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38012)
0809 03:27:50.513439 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.513511 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 03:27:50.513808 23754 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:36045: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
I20260809 03:27:50.529954 23837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } attempt: 1
I20260809 03:27:50.530797 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.530509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38014)
0809 03:27:50.530648 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.530717 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 03:27:50.531561 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.531336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38016)
0809 03:27:50.531443 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.531503 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 03:27:50.531714 23837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:36045: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
W20260809 03:27:50.583374 23846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:50.583639 23846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:50.583719 23846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:50.587697 23846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:27:50.587760 23846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:50.587781 23846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:50.587798 23846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:27:50.587854 23846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:27:50.588986 23754 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } attempt: 2
I20260809 03:27:50.589850 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.589550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38030)
0809 03:27:50.589708 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.589772 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 03:27:50.590611 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.590363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38038)
0809 03:27:50.590489 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.590540 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 03:27:50.590770 23754 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:36045: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
I20260809 03:27:50.593797 23846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:41715,127.9.46.253:37469,127.9.46.252:36045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab
--principal=kudu/127.9.46.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.252:36045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.46.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:50.594866 23846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:50.595870 23846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:27:50.599872 23837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } attempt: 2
I20260809 03:27:50.600772 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.600441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38048)
0809 03:27:50.600612 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.600682 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 03:27:50.601490 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.601265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38058)
0809 03:27:50.601374 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.601429 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 03:27:50.601656 23837 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:36045: Network error: Client connection negotiation failed: client connection to 127.9.46.252:36045: connect: Connection refused (error 111)
W20260809 03:27:50.604761 23858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:50.604805 23846 server_base.cc:1034] running on GCE node
W20260809 03:27:50.604764 23854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:50.604761 23855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:50.605655 23846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:50.606699 23846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:50.607861 23846 hybrid_clock.cc:648] HybridClock initialized: now 1786246070607845 us; error 28 us; skew 500 ppm
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:50.610705 23846 init.cc:377] Logged in from keytab as kudu/127.9.46.252@KRBTEST.COM (short username kudu)
I20260809 03:27:50.611564 23846 webserver.cc:460] Webserver started at http://127.9.46.252:44673/ using document root <none> and password file <none>
I20260809 03:27:50.612074 23846 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:50.612135 23846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:50.612296 23846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:27:50.614379 23846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/data/instance:
uuid: "e63747bb29594fe8aef1fdc3abac9cd7"
format_stamp: "Formatted at 2026-08-09 03:27:50 on dist-test-slave-65mx"
server_key: "b86aac485fa93236cc2ee545de6746b3"
server_key_iv: "dfb3d575b8a10f32bf028cf300e1d4d6"
server_key_version: "encryptionkey@0"
I20260809 03:27:50.614938 23846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/wal/instance:
uuid: "e63747bb29594fe8aef1fdc3abac9cd7"
format_stamp: "Formatted at 2026-08-09 03:27:50 on dist-test-slave-65mx"
server_key: "b86aac485fa93236cc2ee545de6746b3"
server_key_iv: "dfb3d575b8a10f32bf028cf300e1d4d6"
server_key_version: "encryptionkey@0"
I20260809 03:27:50.618676 23846 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:27:50.621043 23865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:50.622246 23846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:27:50.622380 23846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/wal
uuid: "e63747bb29594fe8aef1fdc3abac9cd7"
format_stamp: "Formatted at 2026-08-09 03:27:50 on dist-test-slave-65mx"
server_key: "b86aac485fa93236cc2ee545de6746b3"
server_key_iv: "dfb3d575b8a10f32bf028cf300e1d4d6"
server_key_version: "encryptionkey@0"
I20260809 03:27:50.622488 23846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:50.644659 23846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:50.648936 23846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:50.649165 23846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:50.656687 23846 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.252:36045
I20260809 03:27:50.656766 23921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.252:36045 every 8 connection(s)
I20260809 03:27:50.657696 23846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
I20260809 03:27:50.660419 23923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:50.664078 23923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:27:50.666726 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 23846
I20260809 03:27:50.666868 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786245904130257-9403-0/minicluster-data/master-2/wal/instance
I20260809 03:27:50.667167 9403 external_mini_cluster.cc:1428] Setting key 924086627583181ce604cf6ff44d6c99
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
I20260809 03:27:50.678313 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.665969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45802 (local address 127.9.46.254:41715)
0809 03:27:50.666097 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.666101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.666986 (+ 885us) server_negotiation.cc:407] Connection header received
0809 03:27:50.668060 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.668064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.668162 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.668224 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.669772 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.670279 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.671400 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.671697 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.671778 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.674389 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.674392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.674394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.674419 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.676041 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.676709 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.676711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.676713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.676787 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.677228 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.677232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.677233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.677434 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.677544 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.678019 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.678142 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:27:50.679131 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.667828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38068 (local address 127.9.46.252:36045)
0809 03:27:50.668439 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.668455 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.668476 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:27:50.668603 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.668608 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.668812 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.668998 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.669714 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.670716 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.671593 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.672052 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.672157 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.674802 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.674807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.674816 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.674837 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.676627 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.677106 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.677111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.677112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.677199 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.677484 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.677489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.677491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.677783 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.677983 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.678226 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.678369 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":521,"spinlock_wait_cycles":12032,"thread_start_us":96,"threads_started":1}
I20260809 03:27:50.679136 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.666305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41715 (local address 127.0.0.1:45802)
0809 03:27:50.666823 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.666883 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:50.667806 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:50.668627 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:50.668636 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:50.669097 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:50.669593 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.669607 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.670407 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.670411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.671278 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.671287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.671830 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.671834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.671992 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.672707 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:50.672722 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:50.674236 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:50.676152 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.676162 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.676170 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.676603 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.676891 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.676895 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.676897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.677113 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.677545 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:50.677552 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:50.677795 (+ 243us) client_negotiation.cc:769] Sending connection context
0809 03:27:50.677957 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 03:27:50.678262 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":4,"thread_start_us":171,"threads_started":1}
I20260809 03:27:50.681006 23923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:50.689754 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.681487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:37469 (local address 127.0.0.1:48290)
0809 03:27:50.681729 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.681760 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:50.681828 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:50.682118 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:50.682120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:50.682256 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:50.682346 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.682351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.683164 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.683167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.683685 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.683693 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.684230 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.684232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.684308 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.684925 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:50.684936 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:50.686143 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:50.688139 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.688145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.688148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.688392 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.688725 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.688728 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.688730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.688810 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.689254 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:50.689257 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:50.689345 (+ 88us) client_negotiation.cc:769] Sending connection context
0809 03:27:50.689453 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 03:27:50.689589 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181}
I20260809 03:27:50.689854 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.681750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.9.46.253:37469)
0809 03:27:50.681874 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.681878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.681907 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:27:50.681951 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.681954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.682006 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.682097 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.682460 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.683038 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.683802 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.684114 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.684216 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.686281 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.686284 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.686286 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.686309 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.688015 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.688489 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.688493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.688495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.688630 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.688904 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.688908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.688909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.689156 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.689237 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.689535 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.689670 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:27:50.690001 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.681501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.9.46.254:41715)
0809 03:27:50.681607 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.681611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.681661 (+ 50us) server_negotiation.cc:407] Connection header received
0809 03:27:50.681791 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.681796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.681840 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.681917 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.682612 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.683066 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.683832 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.684031 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.684105 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.686837 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.686840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.686843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.686867 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.688221 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.688705 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.688709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.688711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.688802 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.689111 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.689114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.689115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.689327 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.689557 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.689716 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.689829 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 03:27:50.690037 23754 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } attempt: 3
I20260809 03:27:50.690603 23923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:27:50.692826 23837 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } attempt: 3
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
Aug 09 03:27:50 dist-test-slave-65mx krb5kdc[23664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
I20260809 03:27:50.702067 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.691072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38088)
0809 03:27:50.691410 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.691424 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:50.691493 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:50.691768 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:50.691771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:50.691941 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:50.692034 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.692040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.693687 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.693690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.694437 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.694449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.695058 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.695061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.695143 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.695598 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:50.695612 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:50.697746 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:50.700200 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.700203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.700206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.700583 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.700890 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.700893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.700894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.700989 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.701521 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:50.701523 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:50.701615 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 03:27:50.701720 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 03:27:50.701856 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277}
I20260809 03:27:50.702705 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.691089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38088 (local address 127.9.46.252:36045)
0809 03:27:50.691463 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.691475 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.691489 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:27:50.691565 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.691567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.691663 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.691749 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.692159 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.693563 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.694543 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.694842 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.694948 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.697883 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.697891 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.697897 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.697923 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.700086 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.700690 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.700693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.700695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.700783 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.701098 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.701102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.701104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.701400 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.701490 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.702285 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.702529 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"spinlock_wait_cycles":1792,"thread_start_us":135,"threads_started":1}
I20260809 03:27:50.703061 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.692293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48302 (local address 127.9.46.253:37469)
0809 03:27:50.692404 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.692409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.692424 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:27:50.692464 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.692469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.692517 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.692629 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.693396 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.694386 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.695264 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.695617 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.696061 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.698361 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.698368 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.698373 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.698407 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.700236 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.700780 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.700783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.700786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.700866 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.701211 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.701217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.701219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.701440 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.702617 (+ 1177us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.702736 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.702862 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:27:50.703604 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.690400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38078)
0809 03:27:50.690562 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.690576 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:50.690655 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:50.691097 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:50.691100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:50.691280 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:50.691374 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.691381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.692311 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.692315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.693224 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.693236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.693981 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.693984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.694044 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.695332 (+ 1288us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:50.695348 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:50.697353 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:50.700824 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.700830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.700835 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.701366 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.701927 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.701931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.701933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.702032 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.703088 (+ 1056us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:50.703094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:50.703215 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 03:27:50.703317 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 03:27:50.703456 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 03:27:50.704109 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.690688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38078 (local address 127.9.46.252:36045)
0809 03:27:50.690898 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.690902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.690915 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:27:50.690951 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.690953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.690985 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.691081 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.691520 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.692184 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.693390 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.693740 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.693846 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.697495 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.697499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.697501 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.697526 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.700707 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.701493 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.701496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.701500 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.701580 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.702394 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.702397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.702398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.702700 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.703074 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.703808 (+ 734us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.703918 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260809 03:27:50.705399 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.693555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:36045 (local address 127.0.0.1:38092)
0809 03:27:50.693707 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:50.693722 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:50.693796 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:50.694808 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:50.694813 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:50.694995 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:50.695087 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.695093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.696917 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.696923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.697793 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:50.697806 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.698419 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.698423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:50.698510 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.699093 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:50.699112 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:50.700576 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:50.703533 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.703536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.703538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.703795 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.704272 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:50.704276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:50.704278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:50.704379 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:50.704889 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:50.704892 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:50.704982 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 03:27:50.705109 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 03:27:50.705256 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 03:27:50.706485 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.694023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38092 (local address 127.9.46.252:36045)
0809 03:27:50.694586 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.694596 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.694608 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:27:50.694641 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.694644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.694681 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.694775 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.695210 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.696791 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.697932 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.698311 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.698400 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.700734 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.700740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.700744 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.700781 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.702677 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.703920 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.703923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.703924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.704110 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.704485 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.704489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.704490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.704760 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.706135 (+ 1375us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.706212 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.706325 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"spinlock_wait_cycles":2944,"thread_start_us":105,"threads_started":1}
I20260809 03:27:50.706676 23923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7: Bootstrap starting.
I20260809 03:27:50.708209 23754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804: Bootstrap starting.
I20260809 03:27:50.709079 23923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:50.709064 23837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e: Bootstrap starting.
I20260809 03:27:50.709811 23923 log.cc:824] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:50.710716 23754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:50.711439 23837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:50.711527 23754 log.cc:824] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:50.711731 23923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7: No bootstrap required, opened a new log
I20260809 03:27:50.712183 23837 log.cc:824] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:50.712699 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:27:50.713513 23754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804: No bootstrap required, opened a new log
I20260809 03:27:50.715891 23837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e: No bootstrap required, opened a new log
I20260809 03:27:50.716581 23923 raft_consensus.cc:348] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } }
I20260809 03:27:50.716821 23923 raft_consensus.cc:374] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:50.716881 23923 raft_consensus.cc:729] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63747bb29594fe8aef1fdc3abac9cd7, State: Initialized, Role: FOLLOWER
I20260809 03:27:50.717394 23923 consensus_queue.cc:260] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } }
I20260809 03:27:50.717586 23754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } }
I20260809 03:27:50.717795 23754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:50.717857 23754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 124a20296b374926979568512deb5804, State: Initialized, Role: FOLLOWER
I20260809 03:27:50.718071 23939 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } } }
I20260809 03:27:50.718300 23939 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:27:50.718307 23754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } }
I20260809 03:27:50.718827 23923 sys_catalog.cc:564] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:27:50.718786 23940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } } }
I20260809 03:27:50.719034 23940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 03:27:50.719467 23699 tablet.cc:2369] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:27:50.719537 23754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:27:50.720239 23837 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } }
I20260809 03:27:50.720449 23837 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:50.720510 23837 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec92591f7a814b01b8432a1b82ce4d1e, State: Initialized, Role: FOLLOWER
I20260809 03:27:50.721004 23837 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } }
I20260809 03:27:50.721702 23964 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } } }
I20260809 03:27:50.721881 23964 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:27:50.722790 23837 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [sys.catalog]: configured and running, proceeding with master startup.
W20260809 03:27:50.724464 23962 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:27:50.724593 23962 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:27:50.724745 23677 master_runner.cc:186] Error getting master registration for 127.9.46.253:37469: OK, instance_id { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" instance_seqno: 1786246070435375 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:27:50.730863 23677 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:27:50.732594 23846 master_runner.cc:263] No leader master found from master e63747bb29594fe8aef1fdc3abac9cd7
I20260809 03:27:50.732664 23846 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 03:27:50.734311 23971 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:27:50.734402 23971 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:27:50.734690 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.714826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45818 (local address 127.9.46.254:41715)
0809 03:27:50.714942 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.714947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.724928 (+ 9981us) server_negotiation.cc:407] Connection header received
0809 03:27:50.725447 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.725451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.725512 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.725565 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.726730 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.727657 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.728806 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.729114 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.729208 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.730955 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.730962 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.730965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.730990 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.732661 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.733172 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.733175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.733177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.733267 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.733626 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.733629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.733631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.733888 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.733994 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.734378 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.734520 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:27:50.735267 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.719269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.9.46.253:37469)
0809 03:27:50.719389 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.719393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.725285 (+ 5892us) server_negotiation.cc:407] Connection header received
0809 03:27:50.725963 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.725969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.726031 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.726106 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.726941 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.727426 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.728240 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.728460 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.728591 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.730558 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.730561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.730564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.730589 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.732928 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.733529 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.733533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.733535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.733628 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.733966 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.733969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.733971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.734197 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.734298 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.735001 (+ 703us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.735125 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:27:50.736756 23761 master_runner.cc:263] No leader master found from master ec92591f7a814b01b8432a1b82ce4d1e
I20260809 03:27:50.736826 23761 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 03:27:50.743909 23984 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:27:50.744009 23984 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:27:50.744170 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:50.725678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38096 (local address 127.9.46.252:36045)
0809 03:27:50.728767 (+ 3089us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:50.728774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:50.728788 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:27:50.728825 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:50.728828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:50.728870 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:50.728943 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:50.737410 (+ 8467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.737940 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.738710 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:50.738963 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:50.739065 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:50.740424 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:50.740428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:50.740430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:50.740461 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:50.742235 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.742737 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.742741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.742742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.742831 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:50.743144 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:50.743147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:50.743149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:50.743361 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:50.743478 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:50.743868 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 03:27:50.743999 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2989}
I20260809 03:27:50.887418 23939 raft_consensus.cc:482] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:27:50.887665 23939 raft_consensus.cc:504] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } }
I20260809 03:27:50.888943 23939 leader_election.cc:288] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 124a20296b374926979568512deb5804 (127.9.46.254:41715), ec92591f7a814b01b8432a1b82ce4d1e (127.9.46.253:37469)
I20260809 03:27:50.889721 23723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "124a20296b374926979568512deb5804" is_pre_election: true
I20260809 03:27:50.890059 23723 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63747bb29594fe8aef1fdc3abac9cd7 in term 0.
I20260809 03:27:50.890682 23868 leader_election.cc:302] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 124a20296b374926979568512deb5804, e63747bb29594fe8aef1fdc3abac9cd7; no voters:
I20260809 03:27:50.891060 23939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:27:50.891140 23939 raft_consensus.cc:482] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:27:50.891177 23939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:50.892501 23939 raft_consensus.cc:504] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } }
I20260809 03:27:50.892860 23939 leader_election.cc:288] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [CANDIDATE]: Term 1 election: Requested vote from peers 124a20296b374926979568512deb5804 (127.9.46.254:41715), ec92591f7a814b01b8432a1b82ce4d1e (127.9.46.253:37469)
I20260809 03:27:50.893397 23723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "124a20296b374926979568512deb5804"
I20260809 03:27:50.893546 23723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:50.894758 23723 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63747bb29594fe8aef1fdc3abac9cd7 in term 1.
I20260809 03:27:50.895148 23868 leader_election.cc:302] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 124a20296b374926979568512deb5804, e63747bb29594fe8aef1fdc3abac9cd7; no voters:
I20260809 03:27:50.895357 23939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:27:50.895622 23939 raft_consensus.cc:686] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [term 1 LEADER]: Becoming Leader. State: Replica: e63747bb29594fe8aef1fdc3abac9cd7, State: Running, Role: LEADER
I20260809 03:27:50.895982 23939 consensus_queue.cc:237] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } }
I20260809 03:27:50.896822 23809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" is_pre_election: true
I20260809 03:27:50.897125 23809 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e63747bb29594fe8aef1fdc3abac9cd7 in term 0.
I20260809 03:27:50.897687 23809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec92591f7a814b01b8432a1b82ce4d1e"
I20260809 03:27:50.897792 23809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:50.898913 23809 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e63747bb29594fe8aef1fdc3abac9cd7 in term 1.
W20260809 03:27:50.900599 23871 tablet.cc:2369] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:27:50.901839 23939 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e63747bb29594fe8aef1fdc3abac9cd7. Latest consensus state: current_term: 1 leader_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } } }
I20260809 03:27:50.902025 23939 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:50.904533 23991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:27:50.905330 23991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:27:50.909670 23809 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [term 1 FOLLOWER]: Refusing update from remote peer e63747bb29594fe8aef1fdc3abac9cd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:27:50.909703 23723 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [term 1 FOLLOWER]: Refusing update from remote peer e63747bb29594fe8aef1fdc3abac9cd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:27:50.910339 23939 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:27:50.910529 23989 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:27:50.914225 23964 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e63747bb29594fe8aef1fdc3abac9cd7. Latest consensus state: current_term: 1 leader_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } } }
I20260809 03:27:50.914345 23964 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:27:50.914511 23940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e63747bb29594fe8aef1fdc3abac9cd7. Latest consensus state: current_term: 1 leader_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } } }
I20260809 03:27:50.914633 23940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:27:50.916767 23939 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } } }
I20260809 03:27:50.916877 23939 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:50.917105 23992 mvcc.cc:205] Tried to move back new op lower bound from 7316463906439553024 to 7316463906407567360. Current Snapshot: MvccSnapshot[applied={T|T < 7316463906439553024}]
I20260809 03:27:50.917419 23964 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } } }
I20260809 03:27:50.917542 23964 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:27:50.918284 23993 mvcc.cc:205] Tried to move back new op lower bound from 7316463906439553024 to 7316463906407567360. Current Snapshot: MvccSnapshot[applied={T|T < 7316463906439553024}]
I20260809 03:27:50.919447 23939 sys_catalog.cc:455] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } } }
I20260809 03:27:50.919566 23939 sys_catalog.cc:458] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7 [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:50.920455 23940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "124a20296b374926979568512deb5804" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41715 } } peers { permanent_uuid: "ec92591f7a814b01b8432a1b82ce4d1e" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 37469 } } peers { permanent_uuid: "e63747bb29594fe8aef1fdc3abac9cd7" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 36045 } } }
I20260809 03:27:50.920598 23940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:27:50.922255 23994 mvcc.cc:205] Tried to move back new op lower bound from 7316463906439553024 to 7316463906407567360. Current Snapshot: MvccSnapshot[applied={T|T < 7316463906439553024}]
I20260809 03:27:50.926800 23991 catalog_manager.cc:1347] Generated new cluster ID: 1b58dc354cff493c8852cec349c22741
I20260809 03:27:50.926862 23991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:27:50.952302 23991 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:27:50.953568 23991 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:27:50.961376 23991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e63747bb29594fe8aef1fdc3abac9cd7: Generated new TSK 0
I20260809 03:27:50.962190 23991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:27:51.735803 23971 catalog_manager.cc:1259] Loaded cluster ID: 1b58dc354cff493c8852cec349c22741
I20260809 03:27:51.735915 23971 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804: loading cluster ID for follower catalog manager: success
I20260809 03:27:51.737619 23971 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804: acquiring CA information for follower catalog manager: success
I20260809 03:27:51.738775 23971 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 124a20296b374926979568512deb5804: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:27:51.745123 23984 catalog_manager.cc:1259] Loaded cluster ID: 1b58dc354cff493c8852cec349c22741
I20260809 03:27:51.745186 23984 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e: loading cluster ID for follower catalog manager: success
I20260809 03:27:51.746698 23984 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e: acquiring CA information for follower catalog manager: success
I20260809 03:27:51.747771 23984 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ec92591f7a814b01b8432a1b82ce4d1e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:27:52.018090 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 23677
I20260809 03:27:52.026767 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 23761
I20260809 03:27:52.034443 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 23846
2026-08-09T03:27:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3206 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:27:52 dist-test-slave-65mx krb5kdc[24009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:27:52 dist-test-slave-65mx krb5kdc[24009](info): set up 2 sockets
Aug 09 03:27:52 dist-test-slave-65mx krb5kdc[24009](info): commencing operation
krb5kdc: starting...
W20260809 03:27:53.161394 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:27:53 dist-test-slave-65mx krb5kdc[24009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:27:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:27:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:27:53.289196 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:45413
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:45413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:53.403494 24023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:53.403786 24023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:53.403856 24023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:53.407994 24023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:27:53.408072 24023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:53.408113 24023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:53.408164 24023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:27:53.408205 24023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:27:53.414503 24023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:45413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:45413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:53.415858 24023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:53.417202 24023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:53.426654 24029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:53.426810 24023 server_base.cc:1034] running on GCE node
W20260809 03:27:53.426687 24031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:53.427016 24028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:53.427606 24023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:53.428727 24023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:53.429905 24023 hybrid_clock.cc:648] HybridClock initialized: now 1786246073429886 us; error 30 us; skew 500 ppm
Aug 09 03:27:53 dist-test-slave-65mx krb5kdc[24009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246073, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:53.432761 24023 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:27:53.433697 24023 webserver.cc:460] Webserver started at http://127.9.46.254:43061/ using document root <none> and password file <none>
I20260809 03:27:53.434250 24023 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:53.434340 24023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:53.434564 24023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:27:53.436671 24023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "e291fa859ad64e46838bc7cb54a3ed2d"
format_stamp: "Formatted at 2026-08-09 03:27:53 on dist-test-slave-65mx"
server_key: "e4c2159d9cb427e420eee15a8252e648"
server_key_iv: "887266ddba0a76e272fa0333d91d2faf"
server_key_version: "encryptionkey@0"
I20260809 03:27:53.437260 24023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "e291fa859ad64e46838bc7cb54a3ed2d"
format_stamp: "Formatted at 2026-08-09 03:27:53 on dist-test-slave-65mx"
server_key: "e4c2159d9cb427e420eee15a8252e648"
server_key_iv: "887266ddba0a76e272fa0333d91d2faf"
server_key_version: "encryptionkey@0"
I20260809 03:27:53.441330 24023 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:27:53.444845 24039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:53.446594 24023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:27:53.446741 24023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "e291fa859ad64e46838bc7cb54a3ed2d"
format_stamp: "Formatted at 2026-08-09 03:27:53 on dist-test-slave-65mx"
server_key: "e4c2159d9cb427e420eee15a8252e648"
server_key_iv: "887266ddba0a76e272fa0333d91d2faf"
server_key_version: "encryptionkey@0"
I20260809 03:27:53.446875 24023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:53.462281 24023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:53.465549 24023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:53.465780 24023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:53.475118 24023 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:45413
I20260809 03:27:53.475153 24096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:45413 every 8 connection(s)
I20260809 03:27:53.476027 24023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:27:53.479828 24098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:53.486195 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 24023
I20260809 03:27:53.486336 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:27:53.486694 9403 external_mini_cluster.cc:1428] Setting key cee83fb7b69e0dce0ac4cb70a878cc62
I20260809 03:27:53.487387 24098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d: Bootstrap starting.
I20260809 03:27:53.490360 24098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:53.491343 24098 log.cc:824] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:53.493985 24098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d: No bootstrap required, opened a new log
Aug 09 03:27:53 dist-test-slave-65mx krb5kdc[24009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:53.500020 24098 raft_consensus.cc:348] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291fa859ad64e46838bc7cb54a3ed2d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45413 } }
I20260809 03:27:53.500389 24098 raft_consensus.cc:374] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:53.500459 24098 raft_consensus.cc:729] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e291fa859ad64e46838bc7cb54a3ed2d, State: Initialized, Role: FOLLOWER
I20260809 03:27:53.501055 24098 consensus_queue.cc:260] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291fa859ad64e46838bc7cb54a3ed2d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45413 } }
I20260809 03:27:53.501240 24098 raft_consensus.cc:388] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:27:53.501312 24098 raft_consensus.cc:482] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:27:53.501421 24098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:53.502837 24098 raft_consensus.cc:504] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291fa859ad64e46838bc7cb54a3ed2d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45413 } }
I20260809 03:27:53.503261 24098 leader_election.cc:302] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e291fa859ad64e46838bc7cb54a3ed2d; no voters:
I20260809 03:27:53.503674 24098 leader_election.cc:288] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:27:53.503749 24105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:27:53.503937 24105 raft_consensus.cc:686] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [term 1 LEADER]: Becoming Leader. State: Replica: e291fa859ad64e46838bc7cb54a3ed2d, State: Running, Role: LEADER
I20260809 03:27:53.504230 24105 consensus_queue.cc:237] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291fa859ad64e46838bc7cb54a3ed2d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45413 } }
I20260809 03:27:53.505005 24098 sys_catalog.cc:564] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:27:53.505270 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:53.488512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33662 (local address 127.9.46.254:45413)
0809 03:27:53.489096 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:53.489121 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:53.489174 (+ 53us) server_negotiation.cc:407] Connection header received
0809 03:27:53.490295 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:53.490323 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:53.490799 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:53.491222 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:53.492234 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:53.493790 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:53.495139 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:53.495901 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:53.496086 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:53.499268 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:53.499275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:53.499292 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:53.499327 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:53.501602 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:53.502100 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:53.502106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:53.502112 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:53.502282 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:53.502555 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:53.502560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:53.502563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:53.503111 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:53.503292 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:53.503614 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 03:27:53.504134 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":157,"threads_started":1}
I20260809 03:27:53.505861 24106 sys_catalog.cc:455] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e291fa859ad64e46838bc7cb54a3ed2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291fa859ad64e46838bc7cb54a3ed2d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45413 } } }
I20260809 03:27:53.505995 24106 sys_catalog.cc:458] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:53.506240 24107 sys_catalog.cc:455] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e291fa859ad64e46838bc7cb54a3ed2d. Latest consensus state: current_term: 1 leader_uuid: "e291fa859ad64e46838bc7cb54a3ed2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e291fa859ad64e46838bc7cb54a3ed2d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 45413 } } }
I20260809 03:27:53.506338 24107 sys_catalog.cc:458] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:53.506384 24116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:27:53.508951 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:27:53.510871 24116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:27:53.516426 24116 catalog_manager.cc:1347] Generated new cluster ID: eedbe07a2a3d4780b16b87e9d2daff3c
I20260809 03:27:53.516525 24116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:27:53.525463 24116 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:27:53.526353 24116 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:27:53.535267 24116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e291fa859ad64e46838bc7cb54a3ed2d: Generated new TSK 0
I20260809 03:27:53.535919 24116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:27:53.564881 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 24023
2026-08-09T03:27:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1529 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:27:53 dist-test-slave-65mx krb5kdc[24133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:27:53 dist-test-slave-65mx krb5kdc[24133](info): set up 2 sockets
Aug 09 03:27:53 dist-test-slave-65mx krb5kdc[24133](info): commencing operation
krb5kdc: starting...
W20260809 03:27:54.714118 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:27:54 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:27:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:27:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:27:54.841691 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43177
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:43177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:54.953406 24147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:54.953655 24147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:54.953692 24147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:54.957741 24147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:27:54.957796 24147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:54.957815 24147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:54.957831 24147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:27:54.957877 24147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:27:54.962985 24147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:43177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:54.964012 24147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:54.965098 24147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:54.973312 24153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:54.973366 24154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:54.973333 24157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:54.973699 24147 server_base.cc:1034] running on GCE node
I20260809 03:27:54.974259 24147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:54.975379 24147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:54.976603 24147 hybrid_clock.cc:648] HybridClock initialized: now 1786246074976577 us; error 33 us; skew 500 ppm
Aug 09 03:27:54 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246074, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:54.982870 24147 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:27:54.983777 24147 webserver.cc:460] Webserver started at http://127.9.46.254:44075/ using document root <none> and password file <none>
I20260809 03:27:54.984287 24147 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:54.984347 24147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:54.984508 24147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:27:54.986496 24147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "66c510b2d03349d290d13f0e242dc00b"
format_stamp: "Formatted at 2026-08-09 03:27:54 on dist-test-slave-65mx"
server_key: "f84d529133247b3be60dc27b227fb47c"
server_key_iv: "24452765fee75d15dd8c48cd8484ecad"
server_key_version: "encryptionkey@0"
I20260809 03:27:54.987047 24147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "66c510b2d03349d290d13f0e242dc00b"
format_stamp: "Formatted at 2026-08-09 03:27:54 on dist-test-slave-65mx"
server_key: "f84d529133247b3be60dc27b227fb47c"
server_key_iv: "24452765fee75d15dd8c48cd8484ecad"
server_key_version: "encryptionkey@0"
I20260809 03:27:54.990670 24147 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:27:54.992991 24165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:54.994154 24147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:27:54.994251 24147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "66c510b2d03349d290d13f0e242dc00b"
format_stamp: "Formatted at 2026-08-09 03:27:54 on dist-test-slave-65mx"
server_key: "f84d529133247b3be60dc27b227fb47c"
server_key_iv: "24452765fee75d15dd8c48cd8484ecad"
server_key_version: "encryptionkey@0"
I20260809 03:27:54.994331 24147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:55.008126 24147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:55.011240 24147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:55.011387 24147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:55.018311 24147 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:43177
I20260809 03:27:55.018610 24219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:43177 every 8 connection(s)
I20260809 03:27:55.019323 24147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:27:55.021986 24220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:55.027330 24220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Bootstrap starting.
I20260809 03:27:55.028594 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 24147
I20260809 03:27:55.028738 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:27:55.029059 9403 external_mini_cluster.cc:1428] Setting key d26778bb190e5111cc27e85108559e56
I20260809 03:27:55.030189 24220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:55.030992 24220 log.cc:824] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:55.033018 24220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: No bootstrap required, opened a new log
I20260809 03:27:55.036265 24220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:55.036501 24220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:55.036607 24220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c510b2d03349d290d13f0e242dc00b, State: Initialized, Role: FOLLOWER
I20260809 03:27:55.037070 24220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:55.037220 24220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:27:55.037287 24220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:27:55.037412 24220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 0 FOLLOWER]: Advancing to term 1
Aug 09 03:27:55 dist-test-slave-65mx krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:55.038345 24220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:55.038709 24220 leader_election.cc:302] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66c510b2d03349d290d13f0e242dc00b; no voters:
I20260809 03:27:55.039067 24220 leader_election.cc:288] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:27:55.039129 24225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:27:55.039347 24225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 1 LEADER]: Becoming Leader. State: Replica: 66c510b2d03349d290d13f0e242dc00b, State: Running, Role: LEADER
I20260809 03:27:55.039629 24225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:55.040336 24220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:27:55.041216 24226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66c510b2d03349d290d13f0e242dc00b. Latest consensus state: current_term: 1 leader_uuid: "66c510b2d03349d290d13f0e242dc00b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } } }
I20260809 03:27:55.041304 24227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66c510b2d03349d290d13f0e242dc00b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } } }
I20260809 03:27:55.041442 24227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:55.041472 24226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:55.041844 24235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:27:55.044076 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.030213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.9.46.254:43177)
0809 03:27:55.030654 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:55.030669 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:55.030701 (+ 32us) server_negotiation.cc:407] Connection header received
0809 03:27:55.031449 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:55.031472 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:55.031858 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:55.032184 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:55.033043 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.034029 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.034872 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.035345 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.035512 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.038237 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:55.038249 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:55.038261 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:55.038287 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:55.040349 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:55.040926 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:55.040932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:55.040939 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:55.041095 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:55.041427 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:55.041433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:55.041436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:55.041923 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:55.042137 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:55.042783 (+ 646us) server_negotiation.cc:299] Negotiation successful
0809 03:27:55.043091 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":150,"threads_started":1}
I20260809 03:27:55.045987 24235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:27:55.051499 24235 catalog_manager.cc:1347] Generated new cluster ID: 0ec1792b3e27420f8134e07d5ed8d45a
I20260809 03:27:55.051581 24235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:27:55.083928 24235 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:27:55.084997 24235 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:27:55.090809 24235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Generated new TSK 0
I20260809 03:27:55.091463 24235 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:27:55.127388 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43177
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:55.239331 24249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:55.239619 24249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:55.239663 24249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:55.243942 24249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:55.244002 24249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:55.244066 24249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:27:55.249161 24249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:55.250108 24249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:55.251020 24249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:55.260397 24254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:55.260458 24255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:55.260460 24258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:55.260831 24249 server_base.cc:1034] running on GCE node
I20260809 03:27:55.261233 24249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:55.261864 24249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:55.263046 24249 hybrid_clock.cc:648] HybridClock initialized: now 1786246075263031 us; error 26 us; skew 500 ppm
Aug 09 03:27:55 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:55.265784 24249 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:27:55.266566 24249 webserver.cc:460] Webserver started at http://127.9.46.193:42195/ using document root <none> and password file <none>
I20260809 03:27:55.267058 24249 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:55.267117 24249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:55.267284 24249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:27:55.269306 24249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "1bcb93b82ab84d169631cb3e90675150"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "4fc893dc0bef89b717857bc3e83dc9e4"
server_key_iv: "1f470594b8115ddf4054c5485c1f126e"
server_key_version: "encryptionkey@0"
I20260809 03:27:55.269804 24249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "1bcb93b82ab84d169631cb3e90675150"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "4fc893dc0bef89b717857bc3e83dc9e4"
server_key_iv: "1f470594b8115ddf4054c5485c1f126e"
server_key_version: "encryptionkey@0"
I20260809 03:27:55.273398 24249 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:27:55.275715 24266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:55.276916 24249 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:27:55.277021 24249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "1bcb93b82ab84d169631cb3e90675150"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "4fc893dc0bef89b717857bc3e83dc9e4"
server_key_iv: "1f470594b8115ddf4054c5485c1f126e"
server_key_version: "encryptionkey@0"
I20260809 03:27:55.277099 24249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:55.298805 24249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:55.302208 24249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:55.302414 24249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:55.303025 24249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:55.303889 24249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:55.303957 24249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:55.304025 24249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:55.304071 24249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:55.313207 24249 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:44851
I20260809 03:27:55.313241 24386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:44851 every 8 connection(s)
I20260809 03:27:55.314064 24249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:27:55 dist-test-slave-65mx krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:55.324231 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 24249
I20260809 03:27:55.324381 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:27:55.324736 9403 external_mini_cluster.cc:1428] Setting key 65e2b9f621c5a39d3daf51e9c217e3ce
I20260809 03:27:55.327368 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.315780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:55577 (local address 127.9.46.254:43177)
0809 03:27:55.315918 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:55.315922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:55.316975 (+ 1053us) server_negotiation.cc:407] Connection header received
0809 03:27:55.317821 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:55.317824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:55.317869 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:55.317943 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:55.319224 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.319752 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.320928 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.321169 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.321274 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.323583 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:55.323587 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:55.323590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:55.323617 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:55.325208 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:55.325773 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:55.325776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:55.325778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:55.325864 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:55.326282 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:55.326284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:55.326285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:55.326455 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:55.326534 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:55.327154 (+ 620us) server_negotiation.cc:299] Negotiation successful
0809 03:27:55.327252 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:27:55.328581 24390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.316126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:55577)
0809 03:27:55.316819 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:55.316861 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:55.317649 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:55.318192 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:55.318198 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:55.318572 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:55.319083 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.319097 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.319867 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.319869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.320793 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.320803 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.321314 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.321316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.321455 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.321994 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:55.322016 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:55.323420 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:55.325339 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:55.325345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:55.325358 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:55.325673 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:55.325963 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:55.325970 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:55.325973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:55.326179 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:55.326583 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:55.326589 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:55.326903 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 03:27:55.327109 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 03:27:55.327459 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":118,"threads_started":1}
I20260809 03:27:55.329872 24387 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43177
I20260809 03:27:55.330132 24387 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:55.330585 24387 heartbeater.cc:511] Master 127.9.46.254:43177 requested a full tablet report, sending...
I20260809 03:27:55.332242 24183 ts_manager.cc:194] Registered new tserver with Master: 1bcb93b82ab84d169631cb3e90675150 (127.9.46.193:44851)
I20260809 03:27:55.333390 24183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:55577
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:27:55.354964 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43177
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:55.469301 24393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:55.469545 24393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:55.469586 24393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:55.473480 24393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:55.473531 24393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:55.473587 24393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:27:55.478713 24393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:55.479642 24393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:55.480652 24393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:55.490058 24399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:55.490125 24393 server_base.cc:1034] running on GCE node
W20260809 03:27:55.490038 24398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:55.490015 24401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:55.490625 24393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:55.491268 24393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:55.492460 24393 hybrid_clock.cc:648] HybridClock initialized: now 1786246075492452 us; error 37 us; skew 500 ppm
Aug 09 03:27:55 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:55.495805 24393 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:27:55.496847 24393 webserver.cc:460] Webserver started at http://127.9.46.194:40405/ using document root <none> and password file <none>
I20260809 03:27:55.497539 24393 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:55.497633 24393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:55.497864 24393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:27:55.500030 24393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "a5bfeed579f94839b9c649286f846b9d"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "1bef6d2306aa434c560c38a57e7ff67b"
server_key_iv: "2f749a6a1a88fcf56c7741c7f830c176"
server_key_version: "encryptionkey@0"
I20260809 03:27:55.500687 24393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "a5bfeed579f94839b9c649286f846b9d"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "1bef6d2306aa434c560c38a57e7ff67b"
server_key_iv: "2f749a6a1a88fcf56c7741c7f830c176"
server_key_version: "encryptionkey@0"
I20260809 03:27:55.505414 24393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:27:55.507753 24409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:55.509045 24393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:27:55.509151 24393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "a5bfeed579f94839b9c649286f846b9d"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "1bef6d2306aa434c560c38a57e7ff67b"
server_key_iv: "2f749a6a1a88fcf56c7741c7f830c176"
server_key_version: "encryptionkey@0"
I20260809 03:27:55.509241 24393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:55.533881 24393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:55.536830 24393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:55.536998 24393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:55.537541 24393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:55.538332 24393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:55.538383 24393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:55.538422 24393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:55.538444 24393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:55.547622 24393 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:33443
I20260809 03:27:55.547709 24540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:33443 every 8 connection(s)
I20260809 03:27:55.548532 24393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:27:55.554195 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 24393
I20260809 03:27:55.554324 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:27:55.554580 9403 external_mini_cluster.cc:1428] Setting key 31c547092c8069667c26128f5455dc51
Aug 09 03:27:55 dist-test-slave-65mx krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:55.563997 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.550374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:55323 (local address 127.9.46.254:43177)
0809 03:27:55.550507 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:55.550516 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:55.551356 (+ 840us) server_negotiation.cc:407] Connection header received
0809 03:27:55.552725 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:55.552728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:55.552798 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:55.552889 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:55.554089 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.554729 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.556125 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.556340 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.556421 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.559774 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:55.559780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:55.559787 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:55.559823 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:55.561622 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:55.562258 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:55.562262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:55.562264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:55.562367 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:55.562719 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:55.562724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:55.562730 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:55.563011 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:55.563143 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:55.563649 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 03:27:55.563798 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 03:27:55.564783 24543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.550761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:55323)
0809 03:27:55.551204 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:55.551249 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:55.552427 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:55.553064 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:55.553071 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:55.553418 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:55.553889 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.553902 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.554860 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.554864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.555991 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.556001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.556499 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.556504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.556770 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.557442 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:55.557467 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:55.559579 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:55.561800 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:55.561809 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:55.561825 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:55.562147 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:55.562471 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:55.562474 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:55.562475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:55.562623 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:55.563136 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:55.563141 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:55.563379 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 03:27:55.563557 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 03:27:55.563821 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":108,"threads_started":1}
I20260809 03:27:55.566038 24541 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43177
I20260809 03:27:55.566300 24541 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:27:55.566797 24541 heartbeater.cc:511] Master 127.9.46.254:43177 requested a full tablet report, sending...
I20260809 03:27:55.567835 24183 ts_manager.cc:194] Registered new tserver with Master: a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443)
I20260809 03:27:55.568408 24183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:55323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:27:55.587774 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43177
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:55.698228 24546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:55.698469 24546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:55.698544 24546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:55.702528 24546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:55.702582 24546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:55.702688 24546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:27:55.707782 24546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:43177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:55.708838 24546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:55.709841 24546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:55.719296 24552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:55.719269 24555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:55.719281 24551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:55.719501 24546 server_base.cc:1034] running on GCE node
I20260809 03:27:55.719939 24546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:55.720649 24546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:55.721834 24546 hybrid_clock.cc:648] HybridClock initialized: now 1786246075721820 us; error 25 us; skew 500 ppm
Aug 09 03:27:55 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:55.724829 24546 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:27:55.725701 24546 webserver.cc:460] Webserver started at http://127.9.46.195:34167/ using document root <none> and password file <none>
I20260809 03:27:55.726250 24546 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:55.726337 24546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:55.726559 24546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:27:55.728518 24546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "02429edc078948a59e97eb4c9b923271"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "d4750ef38c7b9a2d87c01270d3b1967a"
server_key_iv: "b3cbdee969d04d8f1180d0f406de485a"
server_key_version: "encryptionkey@0"
I20260809 03:27:55.729120 24546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "02429edc078948a59e97eb4c9b923271"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "d4750ef38c7b9a2d87c01270d3b1967a"
server_key_iv: "b3cbdee969d04d8f1180d0f406de485a"
server_key_version: "encryptionkey@0"
I20260809 03:27:55.733477 24546 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:27:55.735929 24564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:55.737179 24546 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:27:55.737305 24546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "02429edc078948a59e97eb4c9b923271"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "d4750ef38c7b9a2d87c01270d3b1967a"
server_key_iv: "b3cbdee969d04d8f1180d0f406de485a"
server_key_version: "encryptionkey@0"
I20260809 03:27:55.737417 24546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:55.781037 24546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:55.783874 24546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:55.784085 24546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:55.784782 24546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:55.785595 24546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:55.785663 24546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:55.785742 24546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:55.785791 24546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:55.794760 24546 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:43671
I20260809 03:27:55.794826 24698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:43671 every 8 connection(s)
I20260809 03:27:55.795696 24546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:27:55 dist-test-slave-65mx krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:55.804993 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 24546
I20260809 03:27:55.805133 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:27:55.805472 9403 external_mini_cluster.cc:1428] Setting key fe5f24d9a651b007adea385af99bbc50
I20260809 03:27:55.809579 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.797585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:57065 (local address 127.9.46.254:43177)
0809 03:27:55.797724 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:55.797728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:55.798858 (+ 1130us) server_negotiation.cc:407] Connection header received
0809 03:27:55.799779 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:55.799782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:55.799830 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:55.799921 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:55.801141 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.801664 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.802750 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.802954 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.803029 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.805683 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:55.805686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:55.805688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:55.805714 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:55.807465 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:55.808104 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:55.808107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:55.808108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:55.808191 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:55.808486 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:55.808488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:55.808489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:55.808730 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:55.808811 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:55.809355 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 03:27:55.809456 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:27:55.810467 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.797999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:57065)
0809 03:27:55.798715 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:55.798757 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:55.799595 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:55.800085 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:55.800091 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:55.800452 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:55.801005 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.801018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.801769 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.801771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.802644 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.802652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.803088 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.803091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.803226 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.803765 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:55.803781 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:55.805493 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:55.807598 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:55.807606 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:55.807625 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:55.808012 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:55.808269 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:55.808271 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:55.808272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:55.808409 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:55.808835 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:55.808840 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:55.809139 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 03:27:55.809313 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 03:27:55.809579 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":118,"threads_started":1}
I20260809 03:27:55.811534 24699 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43177
I20260809 03:27:55.811754 24699 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:55.812254 24699 heartbeater.cc:511] Master 127.9.46.254:43177 requested a full tablet report, sending...
I20260809 03:27:55.813321 24183 ts_manager.cc:194] Registered new tserver with Master: 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671)
I20260809 03:27:55.813854 24183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:57065
I20260809 03:27:55.819403 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:27:55.828410 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.821466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.9.46.254:43177)
0809 03:27:55.821589 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:55.821592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:55.821790 (+ 198us) server_negotiation.cc:407] Connection header received
0809 03:27:55.821886 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:55.821888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:55.821925 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:55.821990 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:55.822650 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.823138 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.823964 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.824167 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.824239 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.825540 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:55.825542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:55.825544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:55.825567 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:55.826951 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:55.827361 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:55.827364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:55.827365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:55.827442 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:55.827681 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:55.827683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:55.827684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:55.827892 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:55.827993 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:55.828190 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 03:27:55.828292 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 03:27:55.831352 24183 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:27:55.834458 24183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:27:55.848858 24711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.842940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:43671 (local address 127.0.0.1:40838)
0809 03:27:55.843436 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:55.843476 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:55.843579 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:55.844431 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:55.844438 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:55.844469 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:55.844706 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.844717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.846085 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.846090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.847169 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.847180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.848271 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.848274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.848384 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.848425 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:27:55.848579 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 03:27:55.848699 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260809 03:27:55.848858 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.843062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40838 (local address 127.9.46.195:43671)
0809 03:27:55.843881 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:55.843896 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:55.843918 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:27:55.844003 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:55.844008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:55.844228 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:55.844430 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:55.844888 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.845964 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.847314 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.848163 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.848274 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.848426 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:55.848610 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 03:27:55.848708 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":170,"threads_started":1}
I20260809 03:27:55.849838 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.843068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44851 (local address 127.0.0.1:38260)
0809 03:27:55.844420 (+ 1352us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:55.844435 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:55.844928 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:55.845555 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:55.845558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:55.845577 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:55.845658 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.845663 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.847036 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.847040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.848057 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.848072 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.849338 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.849342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.849460 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.849491 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 03:27:55.849581 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:27:55.849688 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1290,"mutex_wait_us":51,"thread_start_us":85,"threads_started":1}
I20260809 03:27:55.849848 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.843766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38260 (local address 127.9.46.193:44851)
0809 03:27:55.844293 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:55.844300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:55.844528 (+ 228us) server_negotiation.cc:407] Connection header received
0809 03:27:55.845071 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:55.845077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:55.845271 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:55.845412 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:55.845789 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.846883 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.848195 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.849047 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.849147 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.849273 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:55.849619 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 03:27:55.849719 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":89,"threads_started":1}
I20260809 03:27:55.850024 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.843905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.9.46.194:33443)
0809 03:27:55.844287 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:55.844300 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:55.844810 (+ 510us) server_negotiation.cc:407] Connection header received
0809 03:27:55.845009 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:55.845015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:55.845182 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:55.845333 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:55.845562 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.846709 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.848120 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.848990 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.849114 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.849262 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:55.849766 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 03:27:55.849861 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":124,"threads_started":1}
I20260809 03:27:55.850147 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:55.843695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:33443 (local address 127.0.0.1:37964)
0809 03:27:55.844708 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:55.844722 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:55.844825 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:55.845319 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:55.845326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:55.845344 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:55.845458 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.845466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.846856 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.846860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.847974 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:55.847985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:55.849328 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:55.849333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:55.849472 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:55.849505 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:27:55.849577 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 03:27:55.849667 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":937,"mutex_wait_us":23,"thread_start_us":101,"threads_started":1}
I20260809 03:27:55.852007 24620 tablet_service.cc:1467] Processing CreateTablet for tablet 4db4a7caecfd479886400d79aaaa05d3 (DEFAULT_TABLE table=test-table [id=a6be305aa8b64a8f9b4b878fff4c2e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:27:55.852294 24317 tablet_service.cc:1467] Processing CreateTablet for tablet 4db4a7caecfd479886400d79aaaa05d3 (DEFAULT_TABLE table=test-table [id=a6be305aa8b64a8f9b4b878fff4c2e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:27:55.853055 24620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db4a7caecfd479886400d79aaaa05d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:55.853271 24317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db4a7caecfd479886400d79aaaa05d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:55.858963 24717 tablet_bootstrap.cc:492] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Bootstrap starting.
I20260809 03:27:55.860273 24465 tablet_service.cc:1467] Processing CreateTablet for tablet 4db4a7caecfd479886400d79aaaa05d3 (DEFAULT_TABLE table=test-table [id=a6be305aa8b64a8f9b4b878fff4c2e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:27:55.860893 24719 tablet_bootstrap.cc:492] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: Bootstrap starting.
I20260809 03:27:55.861094 24717 tablet_bootstrap.cc:654] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:55.861279 24465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db4a7caecfd479886400d79aaaa05d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:55.861734 24717 log.cc:824] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:55.863596 24719 tablet_bootstrap.cc:654] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:55.863673 24717 tablet_bootstrap.cc:492] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: No bootstrap required, opened a new log
I20260809 03:27:55.863863 24717 ts_tablet_manager.cc:1397] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 03:27:55.864451 24719 log.cc:824] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:55.866542 24719 tablet_bootstrap.cc:492] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: No bootstrap required, opened a new log
I20260809 03:27:55.866720 24719 ts_tablet_manager.cc:1397] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 03:27:55.867775 24717 raft_consensus.cc:348] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:55.868005 24717 raft_consensus.cc:374] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:55.868059 24717 raft_consensus.cc:729] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcb93b82ab84d169631cb3e90675150, State: Initialized, Role: FOLLOWER
I20260809 03:27:55.868497 24717 consensus_queue.cc:260] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:55.869251 24387 heartbeater.cc:503] Master 127.9.46.254:43177 was elected leader, sending a full tablet report...
I20260809 03:27:55.869557 24717 ts_tablet_manager.cc:1428] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:27:55.869992 24722 tablet_bootstrap.cc:492] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: Bootstrap starting.
I20260809 03:27:55.870716 24719 raft_consensus.cc:348] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:55.870949 24719 raft_consensus.cc:374] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:55.871023 24719 raft_consensus.cc:729] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02429edc078948a59e97eb4c9b923271, State: Initialized, Role: FOLLOWER
I20260809 03:27:55.871438 24719 consensus_queue.cc:260] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:55.872078 24699 heartbeater.cc:503] Master 127.9.46.254:43177 was elected leader, sending a full tablet report...
I20260809 03:27:55.872399 24719 ts_tablet_manager.cc:1428] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260809 03:27:55.872704 24722 tablet_bootstrap.cc:654] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:55.873582 24722 log.cc:824] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:55.875730 24722 tablet_bootstrap.cc:492] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: No bootstrap required, opened a new log
I20260809 03:27:55.875918 24722 ts_tablet_manager.cc:1397] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:27:55.878783 24722 raft_consensus.cc:348] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:55.879026 24722 raft_consensus.cc:374] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:55.879089 24722 raft_consensus.cc:729] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5bfeed579f94839b9c649286f846b9d, State: Initialized, Role: FOLLOWER
I20260809 03:27:55.879541 24722 consensus_queue.cc:260] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:55.880272 24541 heartbeater.cc:503] Master 127.9.46.254:43177 was elected leader, sending a full tablet report...
I20260809 03:27:55.880661 24722 ts_tablet_manager.cc:1428] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260809 03:27:56.046926 24700 tablet.cc:2369] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:27:56.049734 24542 tablet.cc:2369] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:27:56.065518 24389 tablet.cc:2369] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:27:56.120594 24723 raft_consensus.cc:482] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:27:56.120833 24723 raft_consensus.cc:504] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:56.122030 24723 leader_election.cc:288] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671), a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443)
I20260809 03:27:56.122329 24724 raft_consensus.cc:482] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:27:56.122514 24724 raft_consensus.cc:504] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:56.123732 24724 leader_election.cc:288] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bcb93b82ab84d169631cb3e90675150 (127.9.46.193:44851), a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443)
I20260809 03:27:56.127480 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.122404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:36663 (local address 127.9.46.195:43671)
0809 03:27:56.122543 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:56.122548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:56.122824 (+ 276us) server_negotiation.cc:407] Connection header received
0809 03:27:56.123057 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:56.123061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:56.123114 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:56.123167 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:56.123682 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.124419 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.126195 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.126801 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.126890 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.126938 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:56.127273 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 03:27:56.127342 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:27:56.127872 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.122347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:43671 (local address 127.9.46.193:36663)
0809 03:27:56.122731 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:56.122747 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:56.122917 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:56.123330 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:56.123336 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:56.123358 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:56.123528 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.123566 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.124676 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.124683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.126018 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.126028 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.127032 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.127036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.127132 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.127165 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:27:56.127663 (+ 498us) client_negotiation.cc:240] Negotiation successful
0809 03:27:56.127735 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":135,"threads_started":1}
I20260809 03:27:56.128281 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.122944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:49637 (local address 127.9.46.194:33443)
0809 03:27:56.123127 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:56.123133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:56.123308 (+ 175us) server_negotiation.cc:407] Connection header received
0809 03:27:56.123473 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:56.123476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:56.123551 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:56.123647 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:56.124690 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.125435 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.126952 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.127613 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.127716 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.127761 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:56.128066 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 03:27:56.128134 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":56}
I20260809 03:27:56.128448 24644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db4a7caecfd479886400d79aaaa05d3" candidate_uuid: "1bcb93b82ab84d169631cb3e90675150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02429edc078948a59e97eb4c9b923271" is_pre_election: true
I20260809 03:27:56.128652 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.123971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:33443 (local address 127.9.46.195:36065)
0809 03:27:56.124159 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:56.124175 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:56.124256 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:56.124608 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:56.124614 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:56.124640 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:56.124748 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.124754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.125767 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.125770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.126968 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.126978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.128258 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.128262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.128375 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.128403 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 03:27:56.128457 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 03:27:56.128514 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 03:27:56.128672 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.124399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:50123 (local address 127.9.46.193:44851)
0809 03:27:56.124617 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:56.124623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:56.124705 (+ 82us) server_negotiation.cc:407] Connection header received
0809 03:27:56.124874 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:56.124877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:56.124917 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:56.124962 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:56.125336 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.126029 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.127422 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.127983 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.128076 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.128119 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:56.128447 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 03:27:56.128523 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":127}
I20260809 03:27:56.128674 24644 raft_consensus.cc:2454] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcb93b82ab84d169631cb3e90675150 in term 0.
I20260809 03:27:56.128688 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.122800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:33443 (local address 127.9.46.193:49637)
0809 03:27:56.123190 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:56.123205 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:56.123352 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:56.124375 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:56.124379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:56.124398 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:56.124531 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.124573 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.125577 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.125580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.126807 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.126818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.127819 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.127822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.127918 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.127948 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 03:27:56.128407 (+ 459us) client_negotiation.cc:240] Negotiation successful
0809 03:27:56.128488 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":1,"thread_start_us":81,"threads_started":1}
I20260809 03:27:56.129099 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.124031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44851 (local address 127.9.46.195:50123)
0809 03:27:56.124616 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:56.124630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:56.124769 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:56.125087 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:56.125090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:56.125106 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:56.125214 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.125220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.126172 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.126178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.127302 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.127309 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.128207 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.128210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.128310 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.128329 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 03:27:56.128861 (+ 532us) client_negotiation.cc:240] Negotiation successful
0809 03:27:56.128943 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":118,"threads_started":1}
I20260809 03:27:56.129222 24267 leader_election.cc:302] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02429edc078948a59e97eb4c9b923271, 1bcb93b82ab84d169631cb3e90675150; no voters:
I20260809 03:27:56.129528 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.124016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:36065 (local address 127.9.46.194:33443)
0809 03:27:56.124357 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:56.124361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:56.124372 (+ 11us) server_negotiation.cc:407] Connection header received
0809 03:27:56.124408 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:56.124411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:56.124449 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:56.124844 (+ 395us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:56.124929 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.125649 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.127117 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.128133 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.129124 (+ 991us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.129270 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:56.129304 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 03:27:56.129397 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":261,"thread_start_us":101,"threads_started":1}
I20260809 03:27:56.129518 24723 raft_consensus.cc:2781] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:27:56.129603 24723 raft_consensus.cc:482] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:27:56.129681 24723 raft_consensus.cc:3037] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:56.129606 24339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db4a7caecfd479886400d79aaaa05d3" candidate_uuid: "02429edc078948a59e97eb4c9b923271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcb93b82ab84d169631cb3e90675150" is_pre_election: true
I20260809 03:27:56.129984 24487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db4a7caecfd479886400d79aaaa05d3" candidate_uuid: "1bcb93b82ab84d169631cb3e90675150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5bfeed579f94839b9c649286f846b9d" is_pre_election: true
I20260809 03:27:56.129994 24486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db4a7caecfd479886400d79aaaa05d3" candidate_uuid: "02429edc078948a59e97eb4c9b923271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5bfeed579f94839b9c649286f846b9d" is_pre_election: true
I20260809 03:27:56.130280 24487 raft_consensus.cc:2454] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcb93b82ab84d169631cb3e90675150 in term 0.
I20260809 03:27:56.131063 24723 raft_consensus.cc:504] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:56.131309 24339 raft_consensus.cc:2379] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02429edc078948a59e97eb4c9b923271 in current term 1: Already voted for candidate 1bcb93b82ab84d169631cb3e90675150 in this term.
I20260809 03:27:56.131424 24723 leader_election.cc:288] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 1 election: Requested vote from peers 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671), a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443)
I20260809 03:27:56.132016 24644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db4a7caecfd479886400d79aaaa05d3" candidate_uuid: "1bcb93b82ab84d169631cb3e90675150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02429edc078948a59e97eb4c9b923271"
I20260809 03:27:56.132007 24486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db4a7caecfd479886400d79aaaa05d3" candidate_uuid: "1bcb93b82ab84d169631cb3e90675150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5bfeed579f94839b9c649286f846b9d"
I20260809 03:27:56.132143 24644 raft_consensus.cc:3037] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:56.132153 24486 raft_consensus.cc:3037] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:56.132380 24566 leader_election.cc:302] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02429edc078948a59e97eb4c9b923271; no voters: 1bcb93b82ab84d169631cb3e90675150, a5bfeed579f94839b9c649286f846b9d
I20260809 03:27:56.133257 24644 raft_consensus.cc:2454] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcb93b82ab84d169631cb3e90675150 in term 1.
I20260809 03:27:56.133406 24724 raft_consensus.cc:2726] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 03:27:56.133450 24486 raft_consensus.cc:2454] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcb93b82ab84d169631cb3e90675150 in term 1.
I20260809 03:27:56.133613 24267 leader_election.cc:302] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02429edc078948a59e97eb4c9b923271, 1bcb93b82ab84d169631cb3e90675150; no voters:
I20260809 03:27:56.133810 24723 raft_consensus.cc:2781] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:27:56.134050 24723 raft_consensus.cc:686] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcb93b82ab84d169631cb3e90675150, State: Running, Role: LEADER
I20260809 03:27:56.134348 24723 consensus_queue.cc:237] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:56.137356 24181 catalog_manager.cc:5495] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcb93b82ab84d169631cb3e90675150 (127.9.46.193). New cstate: current_term: 1 leader_uuid: "1bcb93b82ab84d169631cb3e90675150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260809 03:27:56.148114 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.144422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38268 (local address 127.9.46.193:44851)
0809 03:27:56.144615 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:56.144620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:56.144745 (+ 125us) server_negotiation.cc:407] Connection header received
0809 03:27:56.144926 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:56.144929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:56.144988 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:56.145133 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:27:56.145378 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.145882 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.146730 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.146956 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.147069 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.147371 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:27:56.147747 (+ 376us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:27:56.147871 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:56.147915 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 03:27:56.147965 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:27:56.153604 24486 raft_consensus.cc:1261] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d [term 1 FOLLOWER]: Refusing update from remote peer 1bcb93b82ab84d169631cb3e90675150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:27:56.153715 24644 raft_consensus.cc:1261] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 1 FOLLOWER]: Refusing update from remote peer 1bcb93b82ab84d169631cb3e90675150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:27:56.154258 24723 consensus_queue.cc:1035] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:27:56.154450 24731 consensus_queue.cc:1035] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:27:56.161000 24734 mvcc.cc:205] Tried to move back new op lower bound from 7316463927918305280 to 7316463927848472576. Current Snapshot: MvccSnapshot[applied={T|T < 7316463927918305280}]
I20260809 03:27:56.161919 24735 mvcc.cc:205] Tried to move back new op lower bound from 7316463927918305280 to 7316463927848472576. Current Snapshot: MvccSnapshot[applied={T|T < 7316463927918305280}]
I20260809 03:27:56.170114 24183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:27:56.170290 24183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:27:56.172780 24183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Sending DeleteTablet for 3 replicas of tablet 4db4a7caecfd479886400d79aaaa05d3
I20260809 03:27:56.173683 24317 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db4a7caecfd479886400d79aaaa05d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:27:56 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:38260
I20260809 03:27:56.173897 24465 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db4a7caecfd479886400d79aaaa05d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:27:56 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:37964
I20260809 03:27:56.174201 24620 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db4a7caecfd479886400d79aaaa05d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:27:56 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:40838
I20260809 03:27:56.174481 24749 tablet_replica.cc:330] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: stopping tablet replica
I20260809 03:27:56.174664 24750 tablet_replica.cc:330] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: stopping tablet replica
I20260809 03:27:56.174839 24749 raft_consensus.cc:2229] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:27:56.174899 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 24249
I20260809 03:27:56.175174 24749 raft_consensus.cc:2258] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:27:56.176299 24750 raft_consensus.cc:2229] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:27:56.176636 24750 raft_consensus.cc:2258] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:27:56.177906 24749 ts_tablet_manager.cc:1905] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:27:56.178529 24750 ts_tablet_manager.cc:1905] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:27:56.180704 24750 ts_tablet_manager.cc:1918] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:27:56.180814 24750 log.cc:1196] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/4db4a7caecfd479886400d79aaaa05d3
I20260809 03:27:56.181129 24750 ts_tablet_manager.cc:1939] T 4db4a7caecfd479886400d79aaaa05d3 P 02429edc078948a59e97eb4c9b923271: Deleting consensus metadata
I20260809 03:27:56.181516 24749 ts_tablet_manager.cc:1918] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:27:56.181639 24749 log.cc:1196] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/4db4a7caecfd479886400d79aaaa05d3
I20260809 03:27:56.181962 24749 ts_tablet_manager.cc:1939] T 4db4a7caecfd479886400d79aaaa05d3 P a5bfeed579f94839b9c649286f846b9d: Deleting consensus metadata
I20260809 03:27:56.182411 24166 catalog_manager.cc:4841] TS 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671): tablet 4db4a7caecfd479886400d79aaaa05d3 (table test-table [id=a6be305aa8b64a8f9b4b878fff4c2e00]) successfully deleted
I20260809 03:27:56.183193 24166 catalog_manager.cc:4841] TS a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443): tablet 4db4a7caecfd479886400d79aaaa05d3 (table test-table [id=a6be305aa8b64a8f9b4b878fff4c2e00]) successfully deleted
W20260809 03:27:56.185210 24167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:44851 (error 108)
I20260809 03:27:56.185546 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 24393
I20260809 03:27:56.188403 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.188123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44851 (local address 127.0.0.1:38276)
0809 03:27:56.188262 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:56.188328 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 03:27:56.188674 24167 catalog_manager.cc:4584] TS 1bcb93b82ab84d169631cb3e90675150 (127.9.46.193:44851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4db4a7caecfd479886400d79aaaa05d3: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44851: connect: Connection refused (error 111)
I20260809 03:27:56.193470 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 24546
I20260809 03:27:56.200788 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 24147
I20260809 03:27:56.210217 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43177
--webserver_interface=127.9.46.254
--webserver_port=44075
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:43177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:56.320627 24752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:56.320886 24752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:56.320959 24752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:56.324945 24752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:27:56.325009 24752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:56.325029 24752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:56.325045 24752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:27:56.325060 24752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:27:56.330084 24752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:43177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=44075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:56.331010 24752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:56.331913 24752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:56.346926 24758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:56.346926 24759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:56.346984 24763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:56.347004 24752 server_base.cc:1034] running on GCE node
I20260809 03:27:56.347795 24752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:56.349155 24752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:56.350327 24752 hybrid_clock.cc:648] HybridClock initialized: now 1786246076350298 us; error 49 us; skew 500 ppm
Aug 09 03:27:56 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246076, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:56.353080 24752 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:27:56.353915 24752 webserver.cc:460] Webserver started at http://127.9.46.254:44075/ using document root <none> and password file <none>
I20260809 03:27:56.354471 24752 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:56.354547 24752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:56.358410 24752 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:27:56.360538 24771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:56.361790 24752 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:27:56.361896 24752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "66c510b2d03349d290d13f0e242dc00b"
format_stamp: "Formatted at 2026-08-09 03:27:54 on dist-test-slave-65mx"
server_key: "f84d529133247b3be60dc27b227fb47c"
server_key_iv: "24452765fee75d15dd8c48cd8484ecad"
server_key_version: "encryptionkey@0"
I20260809 03:27:56.362468 24752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:56.382845 24752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:56.385685 24752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:56.385844 24752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:56.393003 24752 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:43177
I20260809 03:27:56.393332 24827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:43177 every 8 connection(s)
I20260809 03:27:56.394197 24752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:27:56.397260 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 24752
I20260809 03:27:56.397701 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:44851
--local_ip_for_outbound_sockets=127.9.46.193
--tserver_master_addrs=127.9.46.254:43177
--webserver_port=42195
--webserver_interface=127.9.46.193
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:27:56.398610 24828 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:27:56.400733 24828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Bootstrap starting.
I20260809 03:27:56.418398 24828 log.cc:824] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:56.427004 24828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:27:56.427380 24828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Bootstrap complete.
I20260809 03:27:56.431458 24828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:56.431826 24828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c510b2d03349d290d13f0e242dc00b, State: Initialized, Role: FOLLOWER
I20260809 03:27:56.432332 24828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:56.432443 24828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:27:56.432530 24828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:27:56.432669 24828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:27:56.434494 24828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:56.434810 24828 leader_election.cc:302] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66c510b2d03349d290d13f0e242dc00b; no voters:
I20260809 03:27:56.435103 24828 leader_election.cc:288] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:27:56.435256 24834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:27:56.435511 24834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 2 LEADER]: Becoming Leader. State: Replica: 66c510b2d03349d290d13f0e242dc00b, State: Running, Role: LEADER
I20260809 03:27:56.435812 24834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:56.436282 24828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:27:56.437364 24835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "66c510b2d03349d290d13f0e242dc00b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } } }
I20260809 03:27:56.437407 24836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66c510b2d03349d290d13f0e242dc00b. Latest consensus state: current_term: 2 leader_uuid: "66c510b2d03349d290d13f0e242dc00b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } } }
I20260809 03:27:56.437503 24835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:56.437506 24836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:56.437860 24843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:27:56.442598 24843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:27:56.443657 24843 catalog_manager.cc:1259] Loaded cluster ID: 0ec1792b3e27420f8134e07d5ed8d45a
I20260809 03:27:56.443735 24843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:27:56.445116 24843 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 03:27:56.449566 24843 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 03:27:56.454960 24843 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:27:56.455767 24843 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Loaded TSK: 0
I20260809 03:27:56.457056 24843 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:27:56.519054 24831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:56.519311 24831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:56.519394 24831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:56.523555 24831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:56.523617 24831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:56.523705 24831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:27:56.529475 24831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:44851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=42195
--tserver_master_addrs=127.9.46.254:43177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:56.530489 24831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:56.531457 24831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:56.542009 24862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:56.541997 24860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:56.541997 24865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:56.542726 24831 server_base.cc:1034] running on GCE node
I20260809 03:27:56.543191 24831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:56.543994 24831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:56.545208 24831 hybrid_clock.cc:648] HybridClock initialized: now 1786246076545174 us; error 43 us; skew 500 ppm
Aug 09 03:27:56 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246076, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:56.548764 24831 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:27:56.549886 24831 webserver.cc:460] Webserver started at http://127.9.46.193:42195/ using document root <none> and password file <none>
I20260809 03:27:56.550582 24831 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:56.550688 24831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:56.555823 24831 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:27:56.558531 24873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:56.560032 24831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:27:56.560170 24831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "1bcb93b82ab84d169631cb3e90675150"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "4fc893dc0bef89b717857bc3e83dc9e4"
server_key_iv: "1f470594b8115ddf4054c5485c1f126e"
server_key_version: "encryptionkey@0"
I20260809 03:27:56.560906 24831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:56.583221 24831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:56.586786 24831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:56.586977 24831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:56.587869 24831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:56.589251 24880 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:27:56.593344 24831 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:27:56.593428 24831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260809 03:27:56.593497 24831 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:27:56.595578 24831 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:27:56.595630 24831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 03:27:56.596030 24880 tablet_bootstrap.cc:492] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Bootstrap starting.
I20260809 03:27:56.606271 24831 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:44851
I20260809 03:27:56.606731 24997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:44851 every 8 connection(s)
I20260809 03:27:56.607355 24831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:27:56.615979 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 24831
I20260809 03:27:56.616493 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:33443
--local_ip_for_outbound_sockets=127.9.46.194
--tserver_master_addrs=127.9.46.254:43177
--webserver_port=40405
--webserver_interface=127.9.46.194
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:27:56.618677 24880 log.cc:824] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:27:56 dist-test-slave-65mx krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246076, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:56.627811 24880 tablet_bootstrap.cc:492] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:27:56.628293 24880 tablet_bootstrap.cc:492] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Bootstrap complete.
I20260809 03:27:56.628423 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.609681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:36951)
0809 03:27:56.610313 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:56.610367 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:56.611208 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:56.612698 (+ 1490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:56.612705 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:56.613098 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:56.613570 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.613586 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.615817 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.615822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.617386 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.617401 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.618591 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.618606 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.618832 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.619749 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:56.619775 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:56.621718 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:56.624387 (+ 2669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:56.624395 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:56.624411 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:56.624852 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:56.625302 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:56.625308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:56.625312 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:56.625515 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:56.626259 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:56.626268 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:56.626604 (+ 336us) client_negotiation.cc:769] Sending connection context
0809 03:27:56.626829 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 03:27:56.627344 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":160,"threads_started":1}
I20260809 03:27:56.628440 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.609736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:36951 (local address 127.9.46.254:43177)
0809 03:27:56.610138 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:56.610158 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:56.610499 (+ 341us) server_negotiation.cc:407] Connection header received
0809 03:27:56.611635 (+ 1136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:56.611650 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:56.612148 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:56.612644 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:56.613849 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.615575 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.617545 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.618224 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.618386 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.621953 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:56.621961 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:56.621977 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:56.622013 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:56.624235 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:56.625006 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:56.625013 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:56.625020 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:56.625182 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:56.625650 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:56.625653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:56.625655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:56.626119 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:56.626256 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:56.627039 (+ 783us) server_negotiation.cc:299] Negotiation successful
0809 03:27:56.627353 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":137,"threads_started":1}
I20260809 03:27:56.629938 24880 ts_tablet_manager.cc:1397] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Time spent bootstrapping tablet: real 0.034s user 0.021s sys 0.011s
I20260809 03:27:56.631675 24998 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43177
I20260809 03:27:56.631914 24998 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:56.632460 24998 heartbeater.cc:511] Master 127.9.46.254:43177 requested a full tablet report, sending...
I20260809 03:27:56.632838 24880 raft_consensus.cc:348] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:56.633262 24880 raft_consensus.cc:729] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcb93b82ab84d169631cb3e90675150, State: Initialized, Role: FOLLOWER
I20260809 03:27:56.633746 24880 consensus_queue.cc:260] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } } peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } }
I20260809 03:27:56.634691 24880 ts_tablet_manager.cc:1428] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 03:27:56.635113 24790 ts_manager.cc:194] Registered new tserver with Master: 1bcb93b82ab84d169631cb3e90675150 (127.9.46.193:44851)
I20260809 03:27:56.639181 24790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:36951
I20260809 03:27:56.641710 24998 heartbeater.cc:503] Master 127.9.46.254:43177 was elected leader, sending a full tablet report...
Aug 09 03:27:56 dist-test-slave-65mx krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246076, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:27:56.649981 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.638536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.9.46.193:44851)
0809 03:27:56.639677 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:56.639685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:56.639702 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:27:56.639754 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:56.639758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:56.639943 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:56.640072 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:56.640972 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.642141 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.643080 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.643600 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.643718 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.646280 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:56.646289 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:56.646298 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:56.646320 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:56.647917 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:56.648448 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:56.648455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:56.648462 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:56.648617 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:56.648924 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:56.648928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:56.648929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:56.649177 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:56.649320 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:56.649671 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 03:27:56.649838 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1061,"thread_start_us":88,"threads_started":1}
I20260809 03:27:56.649974 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.638466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44851 (local address 127.0.0.1:38278)
0809 03:27:56.639249 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:56.639318 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:56.639446 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:56.640225 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:56.640229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:56.640707 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:56.640835 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.640844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.642254 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.642257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.642957 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.642968 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.643721 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.643724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.643842 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.644447 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:56.644467 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:56.646124 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:56.648061 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:56.648071 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:56.648076 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:56.648323 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:56.648729 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:56.648732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:56.648734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:56.648827 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:56.649298 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:56.649304 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:56.649437 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 03:27:56.649580 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 03:27:56.649787 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":127,"threads_started":1}
I20260809 03:27:56.651037 24929 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db4a7caecfd479886400d79aaaa05d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:27:56 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:38278
I20260809 03:27:56.651895 25007 tablet_replica.cc:330] stopping tablet replica
I20260809 03:27:56.652196 25007 raft_consensus.cc:2229] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:27:56.652443 25007 raft_consensus.cc:2258] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:27:56.653483 25007 ts_tablet_manager.cc:1905] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:27:56.657200 25007 ts_tablet_manager.cc:1918] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:27:56.657289 25007 log.cc:1196] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/4db4a7caecfd479886400d79aaaa05d3
I20260809 03:27:56.657568 25007 ts_tablet_manager.cc:1939] T 4db4a7caecfd479886400d79aaaa05d3 P 1bcb93b82ab84d169631cb3e90675150: Deleting consensus metadata
I20260809 03:27:56.659221 24773 catalog_manager.cc:4841] TS 1bcb93b82ab84d169631cb3e90675150 (127.9.46.193:44851): tablet 4db4a7caecfd479886400d79aaaa05d3 (table test-table [id=a6be305aa8b64a8f9b4b878fff4c2e00]) successfully deleted
W20260809 03:27:56.741472 25002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:56.741822 25002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:56.741912 25002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:56.746044 25002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:56.746133 25002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:56.746227 25002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:27:56.752264 25002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:33443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=40405
--tserver_master_addrs=127.9.46.254:43177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:56.753455 25002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:56.754528 25002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:56.764257 25012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:56.764257 25015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:56.764271 25013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:56.764456 25002 server_base.cc:1034] running on GCE node
I20260809 03:27:56.765035 25002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:56.765745 25002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:56.766942 25002 hybrid_clock.cc:648] HybridClock initialized: now 1786246076766917 us; error 49 us; skew 500 ppm
Aug 09 03:27:56 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246076, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:56.769876 25002 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:27:56.770684 25002 webserver.cc:460] Webserver started at http://127.9.46.194:40405/ using document root <none> and password file <none>
I20260809 03:27:56.771193 25002 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:56.771262 25002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:56.775017 25002 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:27:56.777071 25023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:56.778295 25002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:27:56.778420 25002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "a5bfeed579f94839b9c649286f846b9d"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "1bef6d2306aa434c560c38a57e7ff67b"
server_key_iv: "2f749a6a1a88fcf56c7741c7f830c176"
server_key_version: "encryptionkey@0"
I20260809 03:27:56.779012 25002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:56.801431 25002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:56.804491 25002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:56.804738 25002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:56.805373 25002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:56.806219 25002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:56.806268 25002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:56.806334 25002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:56.806371 25002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:56.815514 25002 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:33443
I20260809 03:27:56.815605 25147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:33443 every 8 connection(s)
I20260809 03:27:56.816651 25002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:27:56.825845 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 25002
I20260809 03:27:56.826440 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:43671
--local_ip_for_outbound_sockets=127.9.46.195
--tserver_master_addrs=127.9.46.254:43177
--webserver_port=34167
--webserver_interface=127.9.46.195
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 03:27:56 dist-test-slave-65mx krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246076, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:56.831025 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.818694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:53589 (local address 127.9.46.254:43177)
0809 03:27:56.818838 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:56.818842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:56.819931 (+ 1089us) server_negotiation.cc:407] Connection header received
0809 03:27:56.821058 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:56.821062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:56.821128 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:56.821237 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:56.822465 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.823016 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.824194 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.824446 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.824606 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.827335 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:56.827338 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:56.827340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:56.827367 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:56.828898 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:56.829430 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:56.829435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:56.829437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:56.829523 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:56.829876 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:56.829880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:56.829882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:56.830070 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:56.830181 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:56.830701 (+ 520us) server_negotiation.cc:299] Negotiation successful
0809 03:27:56.830849 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:27:56.832031 25150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:56.819083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:53589)
0809 03:27:56.819777 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:56.819820 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:56.820840 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:56.821430 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:56.821437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:56.821807 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:56.822306 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.822320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.823161 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.823164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.824052 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:56.824062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:56.824636 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:56.824642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:56.824844 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:56.825445 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:56.825462 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:56.827131 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:56.829025 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:56.829031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:56.829047 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:56.829321 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:56.829619 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:56.829622 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:56.829624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:56.829775 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:56.830186 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:56.830195 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:56.830452 (+ 257us) client_negotiation.cc:769] Sending connection context
0809 03:27:56.830652 (+ 200us) client_negotiation.cc:240] Negotiation successful
0809 03:27:56.831000 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":162,"threads_started":1}
I20260809 03:27:56.833364 25148 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43177
I20260809 03:27:56.833606 25148 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:56.834115 25148 heartbeater.cc:511] Master 127.9.46.254:43177 requested a full tablet report, sending...
I20260809 03:27:56.835208 24790 ts_manager.cc:194] Registered new tserver with Master: a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443)
I20260809 03:27:56.835799 24790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:53589
W20260809 03:27:56.942484 25151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:56.942746 25151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:56.942791 25151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:56.946944 25151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:56.947003 25151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:56.947067 25151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:27:56.952315 25151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:43671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=34167
--tserver_master_addrs=127.9.46.254:43177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:56.953495 25151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:56.954496 25151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:56.964092 25157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:56.964063 25156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:56.964246 25151 server_base.cc:1034] running on GCE node
W20260809 03:27:56.964044 25160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:56.964766 25151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:56.965539 25151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:56.966702 25151 hybrid_clock.cc:648] HybridClock initialized: now 1786246076966682 us; error 30 us; skew 500 ppm
Aug 09 03:27:56 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246076, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:56.969480 25151 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:27:56.970310 25151 webserver.cc:460] Webserver started at http://127.9.46.195:34167/ using document root <none> and password file <none>
I20260809 03:27:56.970870 25151 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:56.970970 25151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:56.974725 25151 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:27:56.976814 25168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:56.978036 25151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:27:56.978173 25151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "02429edc078948a59e97eb4c9b923271"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "d4750ef38c7b9a2d87c01270d3b1967a"
server_key_iv: "b3cbdee969d04d8f1180d0f406de485a"
server_key_version: "encryptionkey@0"
I20260809 03:27:56.978691 25151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:56.989279 25151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:56.992003 25151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:56.992211 25151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:56.992852 25151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:56.993688 25151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:56.993743 25151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:56.993814 25151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:56.993873 25151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:57.002821 25151 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:43671
I20260809 03:27:57.002844 25293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:43671 every 8 connection(s)
I20260809 03:27:57.003784 25151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:27:57.004187 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 25151
I20260809 03:27:57.013836 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.005037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42356 (local address 127.9.46.254:43177)
0809 03:27:57.005185 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:57.005189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:57.005350 (+ 161us) server_negotiation.cc:407] Connection header received
0809 03:27:57.005484 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:57.005486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:57.005531 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:57.005610 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:57.006523 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.007255 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.008201 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.008598 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.008704 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.010279 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:57.010282 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:57.010284 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:57.010306 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:57.011902 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:57.012386 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:57.012389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:57.012390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:57.012466 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:57.012790 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:57.012793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:57.012795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:57.013004 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:57.013223 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:57.013528 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 03:27:57.013651 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 03:27:57 dist-test-slave-65mx krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246076, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:27:57.019019 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.005922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:43693 (local address 127.9.46.254:43177)
0809 03:27:57.006127 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:57.006130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:57.007427 (+ 1297us) server_negotiation.cc:407] Connection header received
0809 03:27:57.008301 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:57.008304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:57.008347 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:57.008417 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:57.009666 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.010428 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.012159 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.012528 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.012653 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.015715 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:57.015719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:57.015723 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:57.015750 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:57.017013 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:57.017511 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:57.017515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:57.017517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:57.017597 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:57.017940 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:57.017945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:57.017946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:57.018115 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:57.018239 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:57.018728 (+ 489us) server_negotiation.cc:299] Negotiation successful
0809 03:27:57.018843 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260809 03:27:57.019825 25299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.006510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:43693)
0809 03:27:57.007254 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.007312 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:57.008065 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:57.008659 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:57.008670 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:57.009057 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:57.009509 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:57.009521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.010558 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.010564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:57.012027 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:57.012036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.012694 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.012697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:57.012906 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.013532 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:57.013552 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:57.015559 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:57.017132 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:57.017138 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:57.017153 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:57.017403 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:57.017695 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:57.017699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:57.017701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:57.017841 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:57.018240 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:57.018247 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:57.018484 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 03:27:57.018672 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 03:27:57.018981 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":119,"threads_started":1}
I20260809 03:27:57.020862 25294 heartbeater.cc:347] Connected to a master server at 127.9.46.254:43177
I20260809 03:27:57.021095 25294 heartbeater.cc:464] Registering TS with master...
I20260809 03:27:57.021577 25294 heartbeater.cc:511] Master 127.9.46.254:43177 requested a full tablet report, sending...
I20260809 03:27:57.022614 24790 ts_manager.cc:194] Registered new tserver with Master: 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671)
I20260809 03:27:57.023172 24790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:43693
I20260809 03:27:57.027413 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:27:57.031863 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.028109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.9.46.254:43177)
0809 03:27:57.028260 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:57.028265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:57.028418 (+ 153us) server_negotiation.cc:407] Connection header received
0809 03:27:57.028628 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:57.028630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:57.028721 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:57.028828 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:27:57.029085 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.029589 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.030465 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.030695 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.030805 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.031120 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:27:57.031469 (+ 349us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:27:57.031601 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:57.031672 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 03:27:57.031728 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:27:57.032510 24790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:27:57.035702 24790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:27:57.043260 24929 tablet_service.cc:1467] Processing CreateTablet for tablet de379b6000454e558b0d9c54891faf8b (DEFAULT_TABLE table=test-table [id=d5ac8734a1b7424a84e74992103838b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:27:57.043942 24929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de379b6000454e558b0d9c54891faf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:57.046916 24880 tablet_bootstrap.cc:492] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Bootstrap starting.
I20260809 03:27:57.047914 24880 tablet_bootstrap.cc:654] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:57.048043 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.042505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:43671 (local address 127.0.0.1:40854)
0809 03:27:57.042865 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.042878 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:57.042970 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:57.043754 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:57.043757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:57.043769 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:57.044006 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:57.044015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.045393 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.045396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:57.046280 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:57.046290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.047536 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.047543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:57.047639 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.047689 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 03:27:57.047794 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 03:27:57.047892 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":108,"thread_start_us":92,"threads_started":1}
I20260809 03:27:57.048317 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.042275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:33443 (local address 127.0.0.1:37974)
0809 03:27:57.042413 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.042428 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:57.042516 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:57.043731 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:57.043737 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:57.043758 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:57.043880 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:57.043888 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.045557 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.045563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:57.046718 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:57.046729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.047928 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.047931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:57.048088 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.048114 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 03:27:57.048168 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 03:27:57.048235 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":3}
I20260809 03:27:57.048408 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.042935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.9.46.195:43671)
0809 03:27:57.043248 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:57.043260 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:57.043289 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:27:57.043353 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:57.043358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:57.043578 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:57.043853 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:57.044155 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.045244 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.046416 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.047377 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.047534 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.047679 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:57.047835 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 03:27:57.047951 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":134,"threads_started":1}
I20260809 03:27:57.048930 25305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.042496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37974 (local address 127.9.46.194:33443)
0809 03:27:57.043240 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:57.043252 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:57.043277 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:27:57.043357 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:57.043362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:57.043564 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:57.043734 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:57.044074 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.045400 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.046868 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.047802 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.048520 (+ 718us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.048703 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:57.048751 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 03:27:57.048820 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":124,"threads_started":1}
I20260809 03:27:57.049635 24880 tablet_bootstrap.cc:492] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: No bootstrap required, opened a new log
I20260809 03:27:57.049736 24880 ts_tablet_manager.cc:1397] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:27:57.050241 24880 raft_consensus.cc:348] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.050384 24880 raft_consensus.cc:374] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:57.050432 24880 raft_consensus.cc:729] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcb93b82ab84d169631cb3e90675150, State: Initialized, Role: FOLLOWER
I20260809 03:27:57.050580 24880 consensus_queue.cc:260] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.050951 24880 ts_tablet_manager.cc:1428] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 03:27:57.052078 25080 tablet_service.cc:1467] Processing CreateTablet for tablet de379b6000454e558b0d9c54891faf8b (DEFAULT_TABLE table=test-table [id=d5ac8734a1b7424a84e74992103838b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:27:57.052078 25221 tablet_service.cc:1467] Processing CreateTablet for tablet de379b6000454e558b0d9c54891faf8b (DEFAULT_TABLE table=test-table [id=d5ac8734a1b7424a84e74992103838b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:27:57.053264 25221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de379b6000454e558b0d9c54891faf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:57.053539 25080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de379b6000454e558b0d9c54891faf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:27:57.062000 25308 tablet_bootstrap.cc:492] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: Bootstrap starting.
I20260809 03:27:57.062521 25309 tablet_bootstrap.cc:492] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: Bootstrap starting.
I20260809 03:27:57.065657 25308 tablet_bootstrap.cc:654] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:57.065694 25309 tablet_bootstrap.cc:654] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: Neither blocks nor log segments found. Creating new log.
I20260809 03:27:57.066637 25309 log.cc:824] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:57.066648 25308 log.cc:824] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:57.068897 25309 tablet_bootstrap.cc:492] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: No bootstrap required, opened a new log
I20260809 03:27:57.068930 25308 tablet_bootstrap.cc:492] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: No bootstrap required, opened a new log
I20260809 03:27:57.069113 25309 ts_tablet_manager.cc:1397] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:27:57.069134 25308 ts_tablet_manager.cc:1397] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 03:27:57.072002 25308 raft_consensus.cc:348] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.072002 25309 raft_consensus.cc:348] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.072273 25308 raft_consensus.cc:374] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:57.072268 25309 raft_consensus.cc:374] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:27:57.072338 25308 raft_consensus.cc:729] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02429edc078948a59e97eb4c9b923271, State: Initialized, Role: FOLLOWER
I20260809 03:27:57.072342 25309 raft_consensus.cc:729] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5bfeed579f94839b9c649286f846b9d, State: Initialized, Role: FOLLOWER
I20260809 03:27:57.072839 25309 consensus_queue.cc:260] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.072849 25308 consensus_queue.cc:260] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.073652 25294 heartbeater.cc:503] Master 127.9.46.254:43177 was elected leader, sending a full tablet report...
I20260809 03:27:57.073839 25148 heartbeater.cc:503] Master 127.9.46.254:43177 was elected leader, sending a full tablet report...
I20260809 03:27:57.073931 25308 ts_tablet_manager.cc:1428] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 03:27:57.074791 25309 ts_tablet_manager.cc:1428] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
W20260809 03:27:57.108958 24999 tablet.cc:2369] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:27:57.115372 25314 raft_consensus.cc:482] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:27:57.115608 25314 raft_consensus.cc:504] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.116863 25314 leader_election.cc:288] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671), 1bcb93b82ab84d169631cb3e90675150 (127.9.46.193:44851)
I20260809 03:27:57.120998 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.117206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:45275 (local address 127.9.46.195:43671)
0809 03:27:57.117404 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:57.117411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:57.117804 (+ 393us) server_negotiation.cc:407] Connection header received
0809 03:27:57.117922 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:57.117925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:57.117982 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:57.118063 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:57.118364 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.118855 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.119891 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.120352 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.120450 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.120490 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:57.120755 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 03:27:57.120861 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 03:27:57.121115 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.117222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:36449 (local address 127.9.46.193:44851)
0809 03:27:57.117350 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:57.117354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:57.117376 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:27:57.117539 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:57.117542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:57.117615 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:57.117721 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:27:57.118101 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.118755 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.119935 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.120519 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.120657 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.120815 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:57.120914 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 03:27:57.120983 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:27:57.121172 25150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.117155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44851 (local address 127.9.46.194:36449)
0809 03:27:57.117274 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.117291 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:57.117384 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:57.117746 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:57.117748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:57.117791 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:57.117948 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:57.117955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.118864 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.118868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:57.119783 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:57.119792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.120655 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.120658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:57.120776 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.120800 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:27:57.120892 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 03:27:57.120962 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":11}
I20260809 03:27:57.120983 25315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.117159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:43671 (local address 127.9.46.194:45275)
0809 03:27:57.117705 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.117720 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:57.117811 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:57.118118 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:57.118120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:57.118141 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:27:57.118257 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:57.118263 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.118963 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.118965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:57.119768 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:57.119777 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.120479 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.120485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:57.120623 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.120651 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 03:27:57.120724 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 03:27:57.120794 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":107,"thread_start_us":166,"threads_started":1}
I20260809 03:27:57.121801 24949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de379b6000454e558b0d9c54891faf8b" candidate_uuid: "a5bfeed579f94839b9c649286f846b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcb93b82ab84d169631cb3e90675150" is_pre_election: true
I20260809 03:27:57.121985 25243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de379b6000454e558b0d9c54891faf8b" candidate_uuid: "a5bfeed579f94839b9c649286f846b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02429edc078948a59e97eb4c9b923271" is_pre_election: true
I20260809 03:27:57.122104 24949 raft_consensus.cc:2454] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5bfeed579f94839b9c649286f846b9d in term 0.
I20260809 03:27:57.122287 25243 raft_consensus.cc:2454] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5bfeed579f94839b9c649286f846b9d in term 0.
I20260809 03:27:57.122648 25025 leader_election.cc:302] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcb93b82ab84d169631cb3e90675150, a5bfeed579f94839b9c649286f846b9d; no voters:
I20260809 03:27:57.122942 25314 raft_consensus.cc:2781] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:27:57.123045 25314 raft_consensus.cc:482] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:27:57.123102 25314 raft_consensus.cc:3037] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:57.124143 25314 raft_consensus.cc:504] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.124511 25314 leader_election.cc:288] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [CANDIDATE]: Term 1 election: Requested vote from peers 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671), 1bcb93b82ab84d169631cb3e90675150 (127.9.46.193:44851)
I20260809 03:27:57.124884 25243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de379b6000454e558b0d9c54891faf8b" candidate_uuid: "a5bfeed579f94839b9c649286f846b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02429edc078948a59e97eb4c9b923271"
I20260809 03:27:57.124922 24949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de379b6000454e558b0d9c54891faf8b" candidate_uuid: "a5bfeed579f94839b9c649286f846b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bcb93b82ab84d169631cb3e90675150"
I20260809 03:27:57.124990 25243 raft_consensus.cc:3037] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:57.125067 24949 raft_consensus.cc:3037] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:27:57.126055 24949 raft_consensus.cc:2454] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5bfeed579f94839b9c649286f846b9d in term 1.
I20260809 03:27:57.126120 25243 raft_consensus.cc:2454] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5bfeed579f94839b9c649286f846b9d in term 1.
I20260809 03:27:57.126368 25025 leader_election.cc:302] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcb93b82ab84d169631cb3e90675150, a5bfeed579f94839b9c649286f846b9d; no voters:
I20260809 03:27:57.126585 25314 raft_consensus.cc:2781] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:27:57.126834 25314 raft_consensus.cc:686] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 1 LEADER]: Becoming Leader. State: Replica: a5bfeed579f94839b9c649286f846b9d, State: Running, Role: LEADER
I20260809 03:27:57.127166 25314 consensus_queue.cc:237] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.130319 24788 catalog_manager.cc:5495] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d reported cstate change: term changed from 0 to 1, leader changed from <none> to a5bfeed579f94839b9c649286f846b9d (127.9.46.194). New cstate: current_term: 1 leader_uuid: "a5bfeed579f94839b9c649286f846b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } health_report { overall_health: UNKNOWN } } }
I20260809 03:27:57.151232 25305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.147527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37988 (local address 127.9.46.194:33443)
0809 03:27:57.147680 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:57.147685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:57.147700 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:27:57.147882 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:57.147884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:57.147944 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:57.148056 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:27:57.148305 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.148882 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.149721 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:57.149942 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:57.150047 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:57.150367 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:27:57.150804 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:27:57.150937 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:57.151001 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 03:27:57.151059 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:27:57.157265 24949 raft_consensus.cc:1261] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Refusing update from remote peer a5bfeed579f94839b9c649286f846b9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:27:57.157269 25243 raft_consensus.cc:1261] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271 [term 1 FOLLOWER]: Refusing update from remote peer a5bfeed579f94839b9c649286f846b9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:27:57.157971 25314 consensus_queue.cc:1035] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:27:57.158218 25316 consensus_queue.cc:1035] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:27:57.164973 25321 mvcc.cc:205] Tried to move back new op lower bound from 7316463932027813888 to 7316463931915644928. Current Snapshot: MvccSnapshot[applied={T|T < 7316463932027813888}]
I20260809 03:27:57.173436 24787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:27:57.173619 24787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:27:57.175944 24787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Sending DeleteTablet for 3 replicas of tablet de379b6000454e558b0d9c54891faf8b
I20260809 03:27:57.176668 25080 tablet_service.cc:1514] Processing DeleteTablet for tablet de379b6000454e558b0d9c54891faf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:27:57 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:37974
I20260809 03:27:57.176826 25221 tablet_service.cc:1514] Processing DeleteTablet for tablet de379b6000454e558b0d9c54891faf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:27:57 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:40854
I20260809 03:27:57.177134 24929 tablet_service.cc:1514] Processing DeleteTablet for tablet de379b6000454e558b0d9c54891faf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:27:57 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:38278
I20260809 03:27:57.177368 25331 tablet_replica.cc:330] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: stopping tablet replica
I20260809 03:27:57.177474 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 24831
I20260809 03:27:57.177516 25330 tablet_replica.cc:330] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: stopping tablet replica
I20260809 03:27:57.179275 25331 raft_consensus.cc:2229] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:27:57.179742 25331 raft_consensus.cc:2258] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:27:57.180830 25330 raft_consensus.cc:2229] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:27:57.181242 25330 raft_consensus.cc:2258] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:27:57.181645 25331 ts_tablet_manager.cc:1905] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:27:57.183041 25330 ts_tablet_manager.cc:1905] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:27:57.185510 25331 ts_tablet_manager.cc:1918] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:27:57.185683 25331 log.cc:1196] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/de379b6000454e558b0d9c54891faf8b
I20260809 03:27:57.185935 25330 ts_tablet_manager.cc:1918] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:27:57.186038 25330 log.cc:1196] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/de379b6000454e558b0d9c54891faf8b
I20260809 03:27:57.186246 25331 ts_tablet_manager.cc:1939] T de379b6000454e558b0d9c54891faf8b P 02429edc078948a59e97eb4c9b923271: Deleting consensus metadata
I20260809 03:27:57.186378 25330 ts_tablet_manager.cc:1939] T de379b6000454e558b0d9c54891faf8b P a5bfeed579f94839b9c649286f846b9d: Deleting consensus metadata
I20260809 03:27:57.187680 24772 catalog_manager.cc:4841] TS 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671): tablet de379b6000454e558b0d9c54891faf8b (table test-table [id=d5ac8734a1b7424a84e74992103838b2]) successfully deleted
I20260809 03:27:57.187884 24772 catalog_manager.cc:4841] TS a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443): tablet de379b6000454e558b0d9c54891faf8b (table test-table [id=d5ac8734a1b7424a84e74992103838b2]) successfully deleted
W20260809 03:27:57.188840 24773 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:44851 (error 108)
I20260809 03:27:57.189450 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 25002
I20260809 03:27:57.190475 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.189623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44851 (local address 127.0.0.1:38288)
0809 03:27:57.190299 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.190375 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":573}
W20260809 03:27:57.191298 24773 catalog_manager.cc:4584] TS 1bcb93b82ab84d169631cb3e90675150 (127.9.46.193:44851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de379b6000454e558b0d9c54891faf8b: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44851: connect: Connection refused (error 111)
I20260809 03:27:57.197091 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 25151
I20260809 03:27:57.204013 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 24752
I20260809 03:27:57.212327 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43177
--webserver_interface=127.9.46.254
--webserver_port=44075
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:43177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:27:57.321803 25336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:57.322077 25336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:57.322163 25336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:57.326179 25336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:27:57.326237 25336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:57.326257 25336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:57.326319 25336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:27:57.326345 25336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:27:57.331554 25336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:43177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:43177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=44075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:57.332605 25336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:57.333629 25336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:57.342309 25342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:57.342382 25341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:57.342360 25344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:57.343576 25336 server_base.cc:1034] running on GCE node
I20260809 03:27:57.344069 25336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:57.345206 25336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:57.346396 25336 hybrid_clock.cc:648] HybridClock initialized: now 1786246077346380 us; error 34 us; skew 500 ppm
Aug 09 03:27:57 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:57.349113 25336 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:27:57.349977 25336 webserver.cc:460] Webserver started at http://127.9.46.254:44075/ using document root <none> and password file <none>
I20260809 03:27:57.350539 25336 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:57.350636 25336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:57.354404 25336 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 03:27:57.356498 25352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:57.358103 25336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:27:57.358225 25336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "66c510b2d03349d290d13f0e242dc00b"
format_stamp: "Formatted at 2026-08-09 03:27:54 on dist-test-slave-65mx"
server_key: "f84d529133247b3be60dc27b227fb47c"
server_key_iv: "24452765fee75d15dd8c48cd8484ecad"
server_key_version: "encryptionkey@0"
I20260809 03:27:57.358757 25336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:57.384187 25336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:57.387445 25336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:57.387614 25336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:57.394586 25336 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:43177
I20260809 03:27:57.394922 25410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:43177 every 8 connection(s)
I20260809 03:27:57.395654 25336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:27:57.399513 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 25336
I20260809 03:27:57.400010 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:44851
--local_ip_for_outbound_sockets=127.9.46.193
--tserver_master_addrs=127.9.46.254:43177
--webserver_port=42195
--webserver_interface=127.9.46.193
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:27:57.400063 25412 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:27:57.402422 25412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Bootstrap starting.
I20260809 03:27:57.423396 25412 log.cc:824] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:57.434574 25412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:27:57.434995 25412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: Bootstrap complete.
I20260809 03:27:57.439154 25412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:57.439600 25412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c510b2d03349d290d13f0e242dc00b, State: Initialized, Role: FOLLOWER
I20260809 03:27:57.440124 25412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:57.440274 25412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:27:57.440362 25412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:27:57.440478 25412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 2 FOLLOWER]: Advancing to term 3
I20260809 03:27:57.442332 25412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:57.442667 25412 leader_election.cc:302] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66c510b2d03349d290d13f0e242dc00b; no voters:
I20260809 03:27:57.443025 25412 leader_election.cc:288] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 03:27:57.443113 25417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 3 FOLLOWER]: Leader election won for term 3
I20260809 03:27:57.443333 25417 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [term 3 LEADER]: Becoming Leader. State: Replica: 66c510b2d03349d290d13f0e242dc00b, State: Running, Role: LEADER
I20260809 03:27:57.443614 25417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } }
I20260809 03:27:57.444260 25412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:27:57.444588 25419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66c510b2d03349d290d13f0e242dc00b. Latest consensus state: current_term: 3 leader_uuid: "66c510b2d03349d290d13f0e242dc00b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } } }
I20260809 03:27:57.444722 25419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:57.444948 25418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "66c510b2d03349d290d13f0e242dc00b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c510b2d03349d290d13f0e242dc00b" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 43177 } } }
I20260809 03:27:57.445060 25418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b [sys.catalog]: This master's current role is: LEADER
I20260809 03:27:57.449467 25435 catalog_manager.cc:1259] Loaded cluster ID: 0ec1792b3e27420f8134e07d5ed8d45a
I20260809 03:27:57.449534 25435 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: loading cluster ID for follower catalog manager: success
W20260809 03:27:57.450645 25435 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 66c510b2d03349d290d13f0e242dc00b: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260809 03:27:57.450721 25435 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260809 03:27:57.450789 25422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:27:57.453543 25422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:27:57.454093 25422 catalog_manager.cc:1259] Loaded cluster ID: 0ec1792b3e27420f8134e07d5ed8d45a
I20260809 03:27:57.454156 25422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 03:27:57.454805 25422 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ffa9e45b9ad google::LogMessage::Fail() at ??:0
@ 0x7ffa9e45e88a google::LogMessage::SendToLog() at ??:0
@ 0x7ffa9e45b4de google::LogMessage::Flush() at ??:0
@ 0x7ffa9e45ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ffaa8d9efdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ffaa8d9fd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ffaa8d9c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ffaa8de0640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ffaa06ad3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ffaa06adc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ffaa06af3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ffaa069e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ffaa1ce26db start_thread at ??:0
@ 0x7ffa9cf5961f clone at ??:0
W20260809 03:27:57.547113 25414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:57.547366 25414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:57.547441 25414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:57.551597 25414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:57.551654 25414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:57.551744 25414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:27:57.557420 25414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:44851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=42195
--tserver_master_addrs=127.9.46.254:43177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:57.558486 25414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:57.559535 25414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:57.569298 25458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:57.569317 25461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:57.569350 25459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:57.569936 25414 server_base.cc:1034] running on GCE node
I20260809 03:27:57.570346 25414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:57.571080 25414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:57.572263 25414 hybrid_clock.cc:648] HybridClock initialized: now 1786246077572250 us; error 28 us; skew 500 ppm
Aug 09 03:27:57 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:57.575095 25414 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:27:57.576035 25414 webserver.cc:460] Webserver started at http://127.9.46.193:42195/ using document root <none> and password file <none>
I20260809 03:27:57.576649 25414 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:57.576717 25414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:57.580704 25414 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:27:57.582927 25468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:57.584236 25414 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:27:57.584331 25414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "1bcb93b82ab84d169631cb3e90675150"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "4fc893dc0bef89b717857bc3e83dc9e4"
server_key_iv: "1f470594b8115ddf4054c5485c1f126e"
server_key_version: "encryptionkey@0"
I20260809 03:27:57.584965 25414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:57.608623 25414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:57.612042 25414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:57.612257 25414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:57.612986 25414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:57.614140 25478 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:27:57.619704 25414 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:27:57.619767 25414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 03:27:57.619846 25414 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:27:57.621663 25414 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:27:57.621735 25414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260809 03:27:57.622031 25478 tablet_bootstrap.cc:492] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Bootstrap starting.
I20260809 03:27:57.632591 25414 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:44851
I20260809 03:27:57.632707 25599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:44851 every 8 connection(s)
I20260809 03:27:57.633971 25414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:27:57.638110 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 25414
I20260809 03:27:57.638736 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:33443
--local_ip_for_outbound_sockets=127.9.46.194
--tserver_master_addrs=127.9.46.254:43177
--webserver_port=40405
--webserver_interface=127.9.46.194
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:27:57.643543 25478 log.cc:824] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Log is configured to *not* fsync() on all Append() calls
I20260809 03:27:57.650581 25478 tablet_bootstrap.cc:492] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:27:57.650972 25478 tablet_bootstrap.cc:492] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Bootstrap complete.
I20260809 03:27:57.652338 25478 ts_tablet_manager.cc:1397] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Time spent bootstrapping tablet: real 0.031s user 0.018s sys 0.008s
I20260809 03:27:57.655018 25478 raft_consensus.cc:348] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.655714 25478 raft_consensus.cc:729] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcb93b82ab84d169631cb3e90675150, State: Initialized, Role: FOLLOWER
I20260809 03:27:57.656260 25478 consensus_queue.cc:260] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:57.657271 25478 ts_tablet_manager.cc:1428] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150: Time spent starting tablet: real 0.005s user 0.000s sys 0.008s
W20260809 03:27:57.674176 25602 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:27:57.636684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:59167)
0809 03:27:57.637411 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.637475 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:57.638588 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:57.673402 (+ 34814us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":221,"threads_started":1}
W20260809 03:27:57.674830 25600 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:43177 (0 consecutive failures): Network error: Failed to ping master at 127.9.46.254:43177: Client connection negotiation failed: client connection to 127.9.46.254:43177: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:27:57.675575 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.675315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:46143)
0809 03:27:57.675444 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.675517 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 03:27:57.676292 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.676099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:42755)
0809 03:27:57.676201 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.676244 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 03:27:57.676646 25600 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:27:57.760378 25603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:57.760689 25603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:57.760733 25603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:57.764806 25603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:57.764861 25603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:57.764921 25603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:27:57.770095 25603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:33443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=40405
--tserver_master_addrs=127.9.46.254:43177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:57.771136 25603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:57.772137 25603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:57.781420 25613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:57.781416 25610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:57.781525 25611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:57.781699 25603 server_base.cc:1034] running on GCE node
I20260809 03:27:57.782163 25603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:57.782754 25603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:57.783941 25603 hybrid_clock.cc:648] HybridClock initialized: now 1786246077783918 us; error 36 us; skew 500 ppm
Aug 09 03:27:57 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:57.786592 25603 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:27:57.787496 25603 webserver.cc:460] Webserver started at http://127.9.46.194:40405/ using document root <none> and password file <none>
I20260809 03:27:57.788064 25603 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:57.788158 25603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:57.791950 25603 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:27:57.794212 25621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:57.795521 25603 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 03:27:57.795665 25603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "a5bfeed579f94839b9c649286f846b9d"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "1bef6d2306aa434c560c38a57e7ff67b"
server_key_iv: "2f749a6a1a88fcf56c7741c7f830c176"
server_key_version: "encryptionkey@0"
I20260809 03:27:57.796229 25603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:57.809628 25603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:57.812445 25603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:57.812672 25603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:57.813261 25603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:57.814060 25603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:57.814141 25603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:57.814214 25603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:57.814250 25603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:57.823586 25603 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:33443
I20260809 03:27:57.823626 25746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:33443 every 8 connection(s)
I20260809 03:27:57.824538 25603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:27:57.828696 25749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.826598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:40851)
0809 03:27:57.827897 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.828002 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1060,"thread_start_us":180,"threads_started":1}
I20260809 03:27:57.829031 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 25603
W20260809 03:27:57.829159 25747 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:43177 (0 consecutive failures): Network error: Failed to ping master at 127.9.46.254:43177: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
I20260809 03:27:57.829459 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:43671
--local_ip_for_outbound_sockets=127.9.46.195
--tserver_master_addrs=127.9.46.254:43177
--webserver_port=34167
--webserver_interface=127.9.46.195
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:27:57.829852 25749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.829537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:34791)
0809 03:27:57.829736 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.829785 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260809 03:27:57.830461 25749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:57.830248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:48833)
0809 03:27:57.830355 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:57.830402 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 03:27:57.830621 25747 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:27:57.942713 25750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:27:57.942958 25750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:27:57.943010 25750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:27:57.947063 25750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:27:57.947116 25750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:27:57.947173 25750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:27:57.952196 25750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:43671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=34167
--tserver_master_addrs=127.9.46.254:43177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:27:57.953231 25750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:27:57.954200 25750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:27:57.963457 25759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:57.963547 25750 server_base.cc:1034] running on GCE node
W20260809 03:27:57.963482 25756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:27:57.963457 25757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:27:57.964120 25750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:27:57.964773 25750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:27:57.965935 25750 hybrid_clock.cc:648] HybridClock initialized: now 1786246077965920 us; error 26 us; skew 500 ppm
Aug 09 03:27:57 dist-test-slave-65mx krb5kdc[24133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:27:57.968534 25750 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:27:57.969405 25750 webserver.cc:460] Webserver started at http://127.9.46.195:34167/ using document root <none> and password file <none>
I20260809 03:27:57.969952 25750 fs_manager.cc:359] Metadata directory not provided
I20260809 03:27:57.970046 25750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:27:57.973717 25750 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260809 03:27:57.975646 25766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:57.976845 25750 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:27:57.976989 25750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "02429edc078948a59e97eb4c9b923271"
format_stamp: "Formatted at 2026-08-09 03:27:55 on dist-test-slave-65mx"
server_key: "d4750ef38c7b9a2d87c01270d3b1967a"
server_key_iv: "b3cbdee969d04d8f1180d0f406de485a"
server_key_version: "encryptionkey@0"
I20260809 03:27:57.977524 25750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:27:58.008100 25750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:27:58.011047 25750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:27:58.011268 25750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:27:58.011873 25750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:27:58.012768 25750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:27:58.012835 25750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:58.012912 25750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:27:58.012954 25750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:27:58.021986 25750 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:43671
I20260809 03:27:58.022032 25893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:43671 every 8 connection(s)
I20260809 03:27:58.022970 25750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:27:58.026501 25896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:58.025059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:48071)
0809 03:27:58.025709 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:58.025809 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":110,"threads_started":1}
W20260809 03:27:58.026963 25894 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:43177 (0 consecutive failures): Network error: Failed to ping master at 127.9.46.254:43177: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
I20260809 03:27:58.027276 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 25750
I20260809 03:27:58.027578 25896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:58.027350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:40433)
0809 03:27:58.027434 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:58.027490 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 03:27:58.028215 25896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:58.027962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:47467)
0809 03:27:58.028082 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:58.028156 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 03:27:58.028415 25894 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 03:27:58.677716 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:58.677267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:39203)
0809 03:27:58.677540 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:58.677598 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260809 03:27:58.831583 25899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:58.831190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:44875)
0809 03:27:58.831426 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:58.831483 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
I20260809 03:27:59.029529 25900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:59.028998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:33203)
0809 03:27:59.029365 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:59.029425 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":71,"threads_started":1}
I20260809 03:27:59.335606 25901 raft_consensus.cc:482] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:27:59.335913 25901 raft_consensus.cc:504] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:27:59.337138 25901 leader_election.cc:288] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443), 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671)
Aug 09 03:27:59 dist-test-slave-65mx krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.195@KRBTEST.COM
Aug 09 03:27:59 dist-test-slave-65mx krb5kdc[24133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.194@KRBTEST.COM
I20260809 03:27:59.351577 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:59.337781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:43671 (local address 127.9.46.193:40165)
0809 03:27:59.338186 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:59.338202 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:59.338305 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:59.339732 (+ 1427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:59.339738 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:59.340290 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:59.340801 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:59.340816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:59.342083 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:59.342092 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:59.342978 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:59.342988 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:59.343708 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:59.343711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:59.343931 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:59.344696 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:59.344713 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:59.346701 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:59.349085 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:59.349092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:59.349111 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:59.349361 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:59.349758 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:59.349760 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:59.349761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:59.349888 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:59.350654 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:59.350660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:59.350930 (+ 270us) client_negotiation.cc:769] Sending connection context
0809 03:27:59.351110 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 03:27:59.351360 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":136,"spinlock_wait_cycles":20992,"thread_start_us":56,"threads_started":1}
I20260809 03:27:59.351897 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:59.337402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:33443 (local address 127.9.46.193:52685)
0809 03:27:59.338102 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:59.338121 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:27:59.338254 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:27:59.339732 (+ 1478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:27:59.339738 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:27:59.340290 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:27:59.340802 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:59.340818 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:59.342095 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:59.342098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:59.342974 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:27:59.342984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:27:59.343715 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:27:59.343719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:27:59.343924 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:59.344819 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:27:59.344830 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:27:59.347694 (+ 2864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:27:59.350122 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:59.350125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:59.350126 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:59.350303 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:59.350656 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:27:59.350659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:27:59.350661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:27:59.350750 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:27:59.351391 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:27:59.351394 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:27:59.351475 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 03:27:59.351585 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 03:27:59.351728 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":149,"spinlock_wait_cycles":2816,"thread_start_us":85,"threads_started":1}
I20260809 03:27:59.352232 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:59.337581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:52685 (local address 127.9.46.194:33443)
0809 03:27:59.338094 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:59.338110 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:59.338302 (+ 192us) server_negotiation.cc:407] Connection header received
0809 03:27:59.339038 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:59.339044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:59.339380 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:59.339699 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:59.340980 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:59.341929 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:59.343107 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:59.343585 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:59.343772 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:59.347890 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:59.347914 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:59.347934 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:59.347975 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:59.350013 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:59.350413 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:59.350418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:59.350423 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:59.350567 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:59.350845 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:59.350847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:59.350848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:59.351271 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:59.351399 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:59.351748 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 03:27:59.352022 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":133,"threads_started":1}
I20260809 03:27:59.352260 25905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:59.337967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:40165 (local address 127.9.46.195:43671)
0809 03:27:59.338350 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0809 03:27:59.338390 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0809 03:27:59.338435 (+ 45us) server_negotiation.cc:407] Connection header received
0809 03:27:59.339077 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:27:59.339083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:27:59.339380 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:27:59.339698 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:27:59.340984 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:59.341932 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:59.343108 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:27:59.343569 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:27:59.343763 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:27:59.346983 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:27:59.346989 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:27:59.347000 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:27:59.347031 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:27:59.348961 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:59.349482 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:59.349489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:59.349494 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:59.349654 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:27:59.349984 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:27:59.349987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:27:59.349989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:27:59.350513 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:27:59.350677 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 03:27:59.351271 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 03:27:59.351581 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":114,"threads_started":1}
W20260809 03:27:59.354766 25469 leader_election.cc:341] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443): Not found: Tablet not found: de379b6000454e558b0d9c54891faf8b
W20260809 03:27:59.355018 25469 leader_election.cc:341] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671): Not found: Tablet not found: de379b6000454e558b0d9c54891faf8b
I20260809 03:27:59.355087 25469 leader_election.cc:302] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcb93b82ab84d169631cb3e90675150; no voters: 02429edc078948a59e97eb4c9b923271, a5bfeed579f94839b9c649286f846b9d
I20260809 03:27:59.355399 25901 raft_consensus.cc:2726] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:27:59.678817 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:59.678511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:44199)
0809 03:27:59.678670 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:59.678729 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 03:27:59.832770 25906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:27:59.832289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:58999)
0809 03:27:59.832535 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 03:27:59.832659 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260809 03:28:00.030682 25907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:00.030311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:48557)
0809 03:28:00.030550 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:00.030598 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":72,"threads_started":1}
I20260809 03:28:00.680059 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:00.679629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:55675)
0809 03:28:00.679897 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:00.679957 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260809 03:28:00.833933 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:00.833530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:34831)
0809 03:28:00.833781 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:00.833839 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
I20260809 03:28:00.980424 25910 raft_consensus.cc:482] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:28:00.980607 25910 raft_consensus.cc:504] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:28:00.981015 25910 leader_election.cc:288] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443), 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671)
W20260809 03:28:00.981966 25469 leader_election.cc:341] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671): Not found: Tablet not found: de379b6000454e558b0d9c54891faf8b
W20260809 03:28:00.982120 25469 leader_election.cc:341] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443): Not found: Tablet not found: de379b6000454e558b0d9c54891faf8b
I20260809 03:28:00.982209 25469 leader_election.cc:302] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcb93b82ab84d169631cb3e90675150; no voters: 02429edc078948a59e97eb4c9b923271, a5bfeed579f94839b9c649286f846b9d
I20260809 03:28:00.982360 25910 raft_consensus.cc:2726] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:28:01.031832 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:01.031367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:36329)
0809 03:28:01.031675 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:01.031734 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":89,"threads_started":1}
I20260809 03:28:01.681385 25912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:01.680941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:46305)
0809 03:28:01.681228 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:01.681285 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":92,"threads_started":1}
I20260809 03:28:01.835250 25913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:01.834799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:36919)
0809 03:28:01.835098 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:01.835159 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":115,"threads_started":1}
I20260809 03:28:02.033025 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:02.032619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:41221)
0809 03:28:02.032875 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:02.032930 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":87,"threads_started":1}
I20260809 03:28:02.620764 25915 raft_consensus.cc:482] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:28:02.620952 25915 raft_consensus.cc:504] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:28:02.621376 25915 leader_election.cc:288] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443), 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671)
W20260809 03:28:02.622318 25469 leader_election.cc:341] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443): Not found: Tablet not found: de379b6000454e558b0d9c54891faf8b
W20260809 03:28:02.622462 25469 leader_election.cc:341] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671): Not found: Tablet not found: de379b6000454e558b0d9c54891faf8b
I20260809 03:28:02.622521 25469 leader_election.cc:302] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcb93b82ab84d169631cb3e90675150; no voters: 02429edc078948a59e97eb4c9b923271, a5bfeed579f94839b9c649286f846b9d
I20260809 03:28:02.622715 25915 raft_consensus.cc:2726] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:28:02.682849 25917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:02.682351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:48015)
0809 03:28:02.682687 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:02.682749 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":89,"threads_started":1}
I20260809 03:28:02.836675 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:02.836182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:50925)
0809 03:28:02.836460 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:02.836517 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":101,"threads_started":1}
I20260809 03:28:03.034207 25919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:03.033802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:52173)
0809 03:28:03.034061 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:03.034116 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260809 03:28:03.684157 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:03.683706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:34187)
0809 03:28:03.683988 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:03.684046 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
I20260809 03:28:03.837895 25921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:03.837443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:45087)
0809 03:28:03.837727 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:03.837796 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":97,"threads_started":1}
I20260809 03:28:04.035360 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:04.034953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:39715)
0809 03:28:04.035211 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:04.035269 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260809 03:28:04.685437 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:04.684994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:58011)
0809 03:28:04.685271 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:04.685331 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":86,"threads_started":1}
I20260809 03:28:04.839155 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:04.838651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:54981)
0809 03:28:04.838969 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:04.839035 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":95,"threads_started":1}
I20260809 03:28:05.036712 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:05.036156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:58127)
0809 03:28:05.036495 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:05.036616 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":92,"threads_started":1}
I20260809 03:28:05.686794 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:05.686284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:33461)
0809 03:28:05.686610 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:05.686692 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":140,"threads_started":1}
I20260809 03:28:05.840482 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:05.840008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:56575)
0809 03:28:05.840315 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:05.840379 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":105,"threads_started":1}
I20260809 03:28:06.038014 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:06.037582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:54599)
0809 03:28:06.037843 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:06.037906 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":84,"threads_started":1}
I20260809 03:28:06.687978 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:06.687544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:60583)
0809 03:28:06.687810 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:06.687862 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":88,"threads_started":1}
I20260809 03:28:06.841666 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:06.841238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:50657)
0809 03:28:06.841504 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:06.841558 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":91,"threads_started":1}
I20260809 03:28:06.886797 25934 raft_consensus.cc:482] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:28:06.886934 25934 raft_consensus.cc:504] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:28:06.887298 25934 leader_election.cc:288] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443), 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671)
W20260809 03:28:06.888226 25469 leader_election.cc:341] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443): Not found: Tablet not found: de379b6000454e558b0d9c54891faf8b
W20260809 03:28:06.888367 25469 leader_election.cc:341] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671): Not found: Tablet not found: de379b6000454e558b0d9c54891faf8b
I20260809 03:28:06.888432 25469 leader_election.cc:302] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcb93b82ab84d169631cb3e90675150; no voters: 02429edc078948a59e97eb4c9b923271, a5bfeed579f94839b9c649286f846b9d
I20260809 03:28:06.888643 25934 raft_consensus.cc:2726] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:28:07.039263 25935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:07.038825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:47391)
0809 03:28:07.039112 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:07.039171 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":88,"threads_started":1}
I20260809 03:28:07.689312 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:07.688903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:54147)
0809 03:28:07.689160 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:07.689215 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":77,"threads_started":1}
I20260809 03:28:07.842837 25938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:07.842405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:46509)
0809 03:28:07.842683 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:07.842742 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":77,"threads_started":1}
I20260809 03:28:08.040498 25939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:08.040062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:48497)
0809 03:28:08.040331 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:08.040405 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":95,"threads_started":1}
I20260809 03:28:08.690681 25940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:08.690107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:39947)
0809 03:28:08.690511 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:08.690580 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":84,"threads_started":1}
I20260809 03:28:08.844054 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:08.843599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:50221)
0809 03:28:08.843892 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:08.843953 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":79,"threads_started":1}
I20260809 03:28:09.041707 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:09.041285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:56697)
0809 03:28:09.041545 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:09.041599 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260809 03:28:09.075150 25944 raft_consensus.cc:482] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:28:09.075296 25944 raft_consensus.cc:504] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5bfeed579f94839b9c649286f846b9d" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33443 } } peers { permanent_uuid: "02429edc078948a59e97eb4c9b923271" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 43671 } } peers { permanent_uuid: "1bcb93b82ab84d169631cb3e90675150" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44851 } }
I20260809 03:28:09.075806 25944 leader_election.cc:288] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443), 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671)
W20260809 03:28:09.076699 25469 leader_election.cc:341] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5bfeed579f94839b9c649286f846b9d (127.9.46.194:33443): Not found: Tablet not found: de379b6000454e558b0d9c54891faf8b
W20260809 03:28:09.076858 25469 leader_election.cc:341] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 02429edc078948a59e97eb4c9b923271 (127.9.46.195:43671): Not found: Tablet not found: de379b6000454e558b0d9c54891faf8b
I20260809 03:28:09.076934 25469 leader_election.cc:302] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bcb93b82ab84d169631cb3e90675150; no voters: 02429edc078948a59e97eb4c9b923271, a5bfeed579f94839b9c649286f846b9d
I20260809 03:28:09.077126 25944 raft_consensus.cc:2726] T de379b6000454e558b0d9c54891faf8b P 1bcb93b82ab84d169631cb3e90675150 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:28:09.691854 25945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:09.691439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:32795)
0809 03:28:09.691706 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:09.691759 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":100,"threads_started":1}
I20260809 03:28:09.845320 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:09.844904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:57197)
0809 03:28:09.845172 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:09.845226 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":94,"threads_started":1}
I20260809 03:28:10.043026 25947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:10.042623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:59195)
0809 03:28:10.042882 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:10.042936 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":93,"threads_started":1}
I20260809 03:28:10.693091 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:10.692638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:38683)
0809 03:28:10.692904 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:10.692986 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":73,"threads_started":1}
I20260809 03:28:10.846463 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:10.846021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:59327)
0809 03:28:10.846304 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:10.846361 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":76,"threads_started":1}
I20260809 03:28:11.044257 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:11.043763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:36285)
0809 03:28:11.044091 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:11.044155 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":79,"threads_started":1}
I20260809 03:28:11.694284 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:11.693843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:42947)
0809 03:28:11.694131 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:11.694186 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":78,"threads_started":1}
I20260809 03:28:11.847703 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:11.847270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:59671)
0809 03:28:11.847544 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:11.847603 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
I20260809 03:28:12.045521 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:12.045042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.195:33521)
0809 03:28:12.045355 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:12.045413 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":119,"threads_started":1}
I20260809 03:28:12.695629 25955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:12.695208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.193:55327)
0809 03:28:12.695467 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:12.695533 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":90,"threads_started":1}
I20260809 03:28:12.849150 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:12.848730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:43177 (local address 127.9.46.194:43889)
0809 03:28:12.848991 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:12.849048 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:43177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":73,"threads_started":1}
I20260809 03:28:13.028775 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 25414
I20260809 03:28:13.036909 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 25603
I20260809 03:28:13.044687 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 25750
2026-08-09T03:28:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19481 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:28:13 dist-test-slave-65mx krb5kdc[25959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:28:13 dist-test-slave-65mx krb5kdc[25959](info): set up 2 sockets
Aug 09 03:28:13 dist-test-slave-65mx krb5kdc[25959](info): commencing operation
krb5kdc: starting...
W20260809 03:28:14.214447 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:28:14 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:28:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:28:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:28:14.338996 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46637
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:46637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:28:14.449311 25972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:14.449558 25972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:14.449597 25972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:14.453475 25972 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 03:28:14.453523 25972 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 03:28:14.453539 25972 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 03:28:14.453552 25972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:28:14.453570 25972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:14.453588 25972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:14.453604 25972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:28:14.453619 25972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:28:14.458845 25972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:46637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:14.459844 25972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:14.460825 25972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:14.469715 25978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:14.469720 25979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:14.469756 25972 server_base.cc:1034] running on GCE node
W20260809 03:28:14.469720 25981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:14.470602 25972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:14.471647 25972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:14.472841 25972 hybrid_clock.cc:648] HybridClock initialized: now 1786246094472819 us; error 44 us; skew 500 ppm
Aug 09 03:28:14 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246094, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:14.475581 25972 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:28:14.476468 25972 webserver.cc:460] Webserver started at http://127.9.46.254:40929/ using document root <none> and password file <none>
I20260809 03:28:14.477058 25972 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:14.477129 25972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:14.477294 25972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:28:14.479271 25972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "ef06b268ae674451842bef38cfdd3f1d"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "5a604c11b0ceaf364aa244f32728b419"
server_key_iv: "13f26adad7302893de87812fab0d72f8"
server_key_version: "encryptionkey@0"
I20260809 03:28:14.479784 25972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "ef06b268ae674451842bef38cfdd3f1d"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "5a604c11b0ceaf364aa244f32728b419"
server_key_iv: "13f26adad7302893de87812fab0d72f8"
server_key_version: "encryptionkey@0"
I20260809 03:28:14.483418 25972 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 03:28:14.485749 25990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:14.486943 25972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:28:14.487046 25972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "ef06b268ae674451842bef38cfdd3f1d"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "5a604c11b0ceaf364aa244f32728b419"
server_key_iv: "13f26adad7302893de87812fab0d72f8"
server_key_version: "encryptionkey@0"
I20260809 03:28:14.487138 25972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:14.507989 25972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:14.510805 25972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:14.510973 25972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:14.518571 25972 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:46637
I20260809 03:28:14.518900 26047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:46637 every 8 connection(s)
I20260809 03:28:14.519560 25972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:28:14.522418 26048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:28:14.526022 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 25972
I20260809 03:28:14.526209 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:28:14.526497 9403 external_mini_cluster.cc:1428] Setting key 704a663b9ae4851c60886ed90d029e33
I20260809 03:28:14.528471 26048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Bootstrap starting.
I20260809 03:28:14.530660 26048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Neither blocks nor log segments found. Creating new log.
I20260809 03:28:14.531394 26048 log.cc:824] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:28:14.533269 26048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: No bootstrap required, opened a new log
Aug 09 03:28:14 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:28:14.537153 26048 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:14.537453 26048 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:28:14.537550 26048 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef06b268ae674451842bef38cfdd3f1d, State: Initialized, Role: FOLLOWER
I20260809 03:28:14.537972 26048 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:14.538094 26048 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:28:14.538193 26048 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:28:14.538319 26048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:28:14.539208 26048 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:14.539592 26048 leader_election.cc:302] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef06b268ae674451842bef38cfdd3f1d; no voters:
I20260809 03:28:14.539937 26048 leader_election.cc:288] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:28:14.540103 26053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:28:14.540351 26053 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 1 LEADER]: Becoming Leader. State: Replica: ef06b268ae674451842bef38cfdd3f1d, State: Running, Role: LEADER
I20260809 03:28:14.540748 26053 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:14.541324 26048 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:28:14.542362 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:14.527924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.9.46.254:46637)
0809 03:28:14.528397 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:14.528412 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:14.528449 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:28:14.529148 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:14.529165 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:14.529482 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:14.529817 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:14.530667 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:14.531751 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:14.532679 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:14.533150 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:14.533334 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:14.536281 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:14.536286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:14.536300 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:14.536329 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:14.538511 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:14.538960 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:14.538966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:14.538972 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:14.539104 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:14.539427 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:14.539430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:14.539432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:14.539974 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:14.540230 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:14.541008 (+ 778us) server_negotiation.cc:299] Negotiation successful
0809 03:28:14.541324 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":149,"threads_started":1}
I20260809 03:28:14.542680 26055 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef06b268ae674451842bef38cfdd3f1d. Latest consensus state: current_term: 1 leader_uuid: "ef06b268ae674451842bef38cfdd3f1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } } }
I20260809 03:28:14.542764 26055 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: This master's current role is: LEADER
I20260809 03:28:14.542490 26054 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef06b268ae674451842bef38cfdd3f1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } } }
I20260809 03:28:14.543126 26054 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: This master's current role is: LEADER
I20260809 03:28:14.543474 26065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:28:14.546768 26065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:28:14.552112 26065 catalog_manager.cc:1347] Generated new cluster ID: 89ffc2cb4d444f90984b48868228dbcf
I20260809 03:28:14.552183 26065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:28:14.556891 26065 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:28:14.557865 26065 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:28:14.567833 26065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Generated new TSK 0
I20260809 03:28:14.568604 26065 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:28:14.607820 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46637
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:28:14.720170 26077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:14.720398 26077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:14.720474 26077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:14.724478 26077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:14.724529 26077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:14.724655 26077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:28:14.729755 26077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:14.730744 26077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:14.731756 26077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:14.741664 26085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:14.741760 26077 server_base.cc:1034] running on GCE node
W20260809 03:28:14.741688 26083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:14.741686 26082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:14.742467 26077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:14.743149 26077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:14.744349 26077 hybrid_clock.cc:648] HybridClock initialized: now 1786246094744326 us; error 32 us; skew 500 ppm
Aug 09 03:28:14 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246094, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:14.747134 26077 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:28:14.747941 26077 webserver.cc:460] Webserver started at http://127.9.46.193:45709/ using document root <none> and password file <none>
I20260809 03:28:14.748456 26077 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:14.748514 26077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:14.748757 26077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:28:14.750718 26077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "7b657f6beadf4ddba6409b756092968f"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "378a4c859c1bedd0854191c9f52435d8"
server_key_iv: "f93ece1d262adc6b212539a433b7efc2"
server_key_version: "encryptionkey@0"
I20260809 03:28:14.751230 26077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "7b657f6beadf4ddba6409b756092968f"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "378a4c859c1bedd0854191c9f52435d8"
server_key_iv: "f93ece1d262adc6b212539a433b7efc2"
server_key_version: "encryptionkey@0"
I20260809 03:28:14.754818 26077 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:28:14.757197 26093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:14.758343 26077 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:28:14.758463 26077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "7b657f6beadf4ddba6409b756092968f"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "378a4c859c1bedd0854191c9f52435d8"
server_key_iv: "f93ece1d262adc6b212539a433b7efc2"
server_key_version: "encryptionkey@0"
I20260809 03:28:14.758574 26077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:14.772009 26077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:14.774755 26077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:14.774938 26077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:14.775494 26077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:28:14.776271 26077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:28:14.776319 26077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:14.776382 26077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:28:14.776414 26077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:14.785480 26077 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:44377
I20260809 03:28:14.785518 26216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:44377 every 8 connection(s)
I20260809 03:28:14.786336 26077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:28:14.795130 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 26077
I20260809 03:28:14.795253 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:28:14.795568 9403 external_mini_cluster.cc:1428] Setting key 1da066afb631c7faaf6bbbe3df0e1ff2
Aug 09 03:28:14 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246094, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:28:14.800495 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:14.788054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:44841 (local address 127.9.46.254:46637)
0809 03:28:14.788203 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:14.788207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:14.789008 (+ 801us) server_negotiation.cc:407] Connection header received
0809 03:28:14.790355 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:14.790359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:14.790416 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:14.790527 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:14.791750 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:14.792291 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:14.793535 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:14.793783 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:14.793899 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:14.796502 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:14.796513 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:14.796515 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:14.796578 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:14.798294 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:14.798833 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:14.798839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:14.798843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:14.798944 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:14.799280 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:14.799286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:14.799290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:14.799558 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:14.799677 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:14.800204 (+ 527us) server_negotiation.cc:299] Negotiation successful
0809 03:28:14.800333 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 03:28:14.801328 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:14.788391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:44841)
0809 03:28:14.788863 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:14.788904 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:14.790131 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:14.790695 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:14.790701 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:14.791046 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:28:14.791601 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:14.791614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:14.792423 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:14.792427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:14.793392 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:14.793402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:14.793945 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:14.793949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:14.794119 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:14.794686 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:28:14.794703 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:28:14.796335 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:28:14.798429 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:14.798434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:14.798447 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:14.798728 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:14.799040 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:14.799042 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:14.799043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:14.799188 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:14.799681 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:28:14.799690 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:28:14.799952 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 03:28:14.800161 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 03:28:14.800491 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":156,"threads_started":1}
I20260809 03:28:14.802553 26217 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46637
I20260809 03:28:14.802815 26217 heartbeater.cc:464] Registering TS with master...
I20260809 03:28:14.803297 26217 heartbeater.cc:511] Master 127.9.46.254:46637 requested a full tablet report, sending...
I20260809 03:28:14.804888 26007 ts_manager.cc:194] Registered new tserver with Master: 7b657f6beadf4ddba6409b756092968f (127.9.46.193:44377)
I20260809 03:28:14.805997 26007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:44841
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:28:14.828047 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46637
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:28:14.939183 26222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:14.939460 26222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:14.939535 26222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:14.943756 26222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:14.943838 26222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:14.943929 26222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:28:14.949198 26222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:14.950261 26222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:14.951288 26222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:14.960634 26230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:14.960698 26227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:14.960698 26228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:14.960867 26222 server_base.cc:1034] running on GCE node
I20260809 03:28:14.961252 26222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:14.961879 26222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:14.963066 26222 hybrid_clock.cc:648] HybridClock initialized: now 1786246094963061 us; error 30 us; skew 500 ppm
Aug 09 03:28:14 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246094, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:14.965870 26222 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:28:14.966672 26222 webserver.cc:460] Webserver started at http://127.9.46.194:44219/ using document root <none> and password file <none>
I20260809 03:28:14.967172 26222 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:14.967231 26222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:14.967387 26222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:28:14.969403 26222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "7f530e8fdc9d4798bd5d5457642946dc"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "caa211c7dfad659c1809550acc1e1e65"
server_key_iv: "0baba4b889b14d55121bedc6c0460b3a"
server_key_version: "encryptionkey@0"
I20260809 03:28:14.969910 26222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "7f530e8fdc9d4798bd5d5457642946dc"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "caa211c7dfad659c1809550acc1e1e65"
server_key_iv: "0baba4b889b14d55121bedc6c0460b3a"
server_key_version: "encryptionkey@0"
I20260809 03:28:14.973445 26222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:28:14.975694 26237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:14.976933 26222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:28:14.977067 26222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "7f530e8fdc9d4798bd5d5457642946dc"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "caa211c7dfad659c1809550acc1e1e65"
server_key_iv: "0baba4b889b14d55121bedc6c0460b3a"
server_key_version: "encryptionkey@0"
I20260809 03:28:14.977180 26222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:15.013104 26222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:15.016000 26222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:15.016218 26222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:15.016844 26222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:28:15.017712 26222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:28:15.017762 26222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:15.017849 26222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:28:15.017890 26222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:15.027323 26222 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:39467
I20260809 03:28:15.027352 26362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:39467 every 8 connection(s)
I20260809 03:28:15.028225 26222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:28:15.035252 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 26222
I20260809 03:28:15.035413 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:28:15.035764 9403 external_mini_cluster.cc:1428] Setting key e0883bedf5874fb632237f20e634344f
Aug 09 03:28:15 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246094, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:28:15.042279 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.030054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:48281 (local address 127.9.46.254:46637)
0809 03:28:15.030213 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:15.030217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:15.031003 (+ 786us) server_negotiation.cc:407] Connection header received
0809 03:28:15.032349 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:15.032352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:15.032402 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:15.032479 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:15.033641 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.034161 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.035267 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.035589 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.035704 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.038383 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:15.038386 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:15.038388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:15.038411 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:15.039929 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:15.040474 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:15.040482 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:15.040483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:15.040613 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:15.041010 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:15.041016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:15.041021 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:15.041317 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:15.041430 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:15.041972 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 03:28:15.042109 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:28:15.043222 26366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.030407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:48281)
0809 03:28:15.030858 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:15.030900 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:15.032094 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:15.032708 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:15.032715 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:15.033059 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:28:15.033510 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.033522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.034268 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.034271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.035130 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.035138 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.035747 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.035751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.035974 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.036673 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:28:15.036689 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:28:15.038213 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:28:15.040057 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:15.040063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:15.040076 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:15.040354 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:15.040712 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:15.040716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:15.040718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:15.040864 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:15.041441 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:28:15.041446 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:28:15.041664 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 03:28:15.041865 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 03:28:15.042190 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":126,"threads_started":1}
I20260809 03:28:15.044412 26363 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46637
I20260809 03:28:15.044720 26363 heartbeater.cc:464] Registering TS with master...
I20260809 03:28:15.045302 26363 heartbeater.cc:511] Master 127.9.46.254:46637 requested a full tablet report, sending...
I20260809 03:28:15.046411 26007 ts_manager.cc:194] Registered new tserver with Master: 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467)
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:28:15.047009 26007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:48281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:28:15.065323 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46637
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:28:15.177500 26369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:15.177795 26369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:15.177881 26369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:15.182034 26369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:15.182112 26369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:15.182197 26369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:28:15.187446 26369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:15.188637 26369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:15.189640 26369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:15.199394 26375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:15.199420 26378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:15.199419 26376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:15.199465 26369 server_base.cc:1034] running on GCE node
I20260809 03:28:15.200059 26369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:15.200776 26369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:15.201963 26369 hybrid_clock.cc:648] HybridClock initialized: now 1786246095201950 us; error 25 us; skew 500 ppm
Aug 09 03:28:15 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246095, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:15.204694 26369 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:28:15.205472 26369 webserver.cc:460] Webserver started at http://127.9.46.195:35605/ using document root <none> and password file <none>
I20260809 03:28:15.205958 26369 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:15.206013 26369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:15.206178 26369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:28:15.208040 26369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "713e900cfbfe47f0ab21680d78a1d418"
format_stamp: "Formatted at 2026-08-09 03:28:15 on dist-test-slave-65mx"
server_key: "75425b83b82a486aefb2dfd2e8e4ac35"
server_key_iv: "34005c4bf8ecef2814b6ed9bd44093f9"
server_key_version: "encryptionkey@0"
I20260809 03:28:15.208583 26369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "713e900cfbfe47f0ab21680d78a1d418"
format_stamp: "Formatted at 2026-08-09 03:28:15 on dist-test-slave-65mx"
server_key: "75425b83b82a486aefb2dfd2e8e4ac35"
server_key_iv: "34005c4bf8ecef2814b6ed9bd44093f9"
server_key_version: "encryptionkey@0"
I20260809 03:28:15.212016 26369 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:28:15.214360 26386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:15.215518 26369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:28:15.215620 26369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "713e900cfbfe47f0ab21680d78a1d418"
format_stamp: "Formatted at 2026-08-09 03:28:15 on dist-test-slave-65mx"
server_key: "75425b83b82a486aefb2dfd2e8e4ac35"
server_key_iv: "34005c4bf8ecef2814b6ed9bd44093f9"
server_key_version: "encryptionkey@0"
I20260809 03:28:15.215705 26369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:15.230937 26369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:15.233870 26369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:15.234030 26369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:15.234565 26369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:28:15.235327 26369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:28:15.235373 26369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:15.235414 26369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:28:15.235430 26369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:15.244609 26369 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:41861
I20260809 03:28:15.244656 26510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:41861 every 8 connection(s)
I20260809 03:28:15.245474 26369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:28:15.252115 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 26369
I20260809 03:28:15.252238 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:28:15.252532 9403 external_mini_cluster.cc:1428] Setting key 5f6871a992006240c598f5f8c2ce861f
Aug 09 03:28:15 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246095, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:28:15.258720 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.247199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:53569 (local address 127.9.46.254:46637)
0809 03:28:15.247353 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:15.247357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:15.248222 (+ 865us) server_negotiation.cc:407] Connection header received
0809 03:28:15.249593 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:15.249596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:15.249642 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:15.249713 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:15.250828 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.251339 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.252519 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.252846 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.252972 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.255667 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:15.255670 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:15.255672 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:15.255695 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:15.256974 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:15.257447 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:15.257449 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:15.257450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:15.257525 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:15.257817 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:15.257819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:15.257820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:15.257988 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:15.258070 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:15.258503 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 03:28:15.258603 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:28:15.259500 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.247634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:53569)
0809 03:28:15.248071 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:15.248115 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:15.249405 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:15.249865 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:15.249871 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:15.250212 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:28:15.250706 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.250719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.251443 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.251446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.252400 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.252410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.253045 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.253052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.253243 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.253890 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:28:15.253913 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:28:15.255518 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:28:15.257086 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:15.257091 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:15.257101 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:15.257360 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:15.257603 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:15.257605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:15.257607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:15.257739 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:15.258075 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:28:15.258079 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:28:15.258305 (+ 226us) client_negotiation.cc:769] Sending connection context
0809 03:28:15.258454 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 03:28:15.258693 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260809 03:28:15.260654 26511 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46637
I20260809 03:28:15.260892 26511 heartbeater.cc:464] Registering TS with master...
I20260809 03:28:15.261356 26511 heartbeater.cc:511] Master 127.9.46.254:46637 requested a full tablet report, sending...
I20260809 03:28:15.262406 26007 ts_manager.cc:194] Registered new tserver with Master: 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861)
I20260809 03:28:15.262928 26007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:53569
I20260809 03:28:15.266333 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:28:15.276091 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.268291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.9.46.254:46637)
0809 03:28:15.268440 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:15.268444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:15.268680 (+ 236us) server_negotiation.cc:407] Connection header received
0809 03:28:15.268836 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:15.268839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:15.268888 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:15.268977 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:15.269682 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.270188 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.271053 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.271291 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.271399 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.272855 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:15.272859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:15.272861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:15.272893 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:15.274366 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:15.274805 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:15.274809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:15.274810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:15.274892 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:15.275178 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:15.275182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:15.275183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:15.275411 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:15.275555 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:15.275814 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 03:28:15.275928 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:28:15.279207 26007 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:28:15.282449 26007 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:28:15.297145 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.291513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43582 (local address 127.9.46.193:44377)
0809 03:28:15.292254 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:15.292270 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:15.292344 (+ 74us) server_negotiation.cc:407] Connection header received
0809 03:28:15.292425 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:15.292433 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:15.292660 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:15.292802 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:28:15.293056 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.293868 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.295583 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.296234 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.296367 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.296529 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:15.296856 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 03:28:15.296985 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":132,"threads_started":1}
I20260809 03:28:15.297147 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.291379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:39467 (local address 127.0.0.1:32840)
0809 03:28:15.292192 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:15.292205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:15.292291 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:15.292978 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:15.292982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:15.293003 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:28:15.293083 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.293088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.294424 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.294427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.295564 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.295575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.296650 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.296654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.296790 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.296838 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 03:28:15.296903 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 03:28:15.296999 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":33,"thread_start_us":103,"threads_started":1}
I20260809 03:28:15.297154 26523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.290660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:41861 (local address 127.0.0.1:43618)
0809 03:28:15.291700 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:15.291752 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:15.291966 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:15.292654 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:15.292660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:15.292685 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:28:15.292825 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.292832 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.294362 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.294365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.295487 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.295497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.296654 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.296660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.296805 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.296846 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:28:15.296916 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 03:28:15.297003 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":859,"mutex_wait_us":62,"thread_start_us":94,"threads_started":1}
I20260809 03:28:15.297014 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.290783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44377 (local address 127.0.0.1:43582)
0809 03:28:15.291862 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:15.291876 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:15.291972 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:15.292796 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:15.292799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:15.292816 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:28:15.292910 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.292917 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.294037 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.294040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.295433 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.295444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.296380 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.296383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.296496 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.296534 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 03:28:15.296679 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 03:28:15.296816 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"thread_start_us":100,"threads_started":1}
I20260809 03:28:15.297340 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.290794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43618 (local address 127.9.46.195:41861)
0809 03:28:15.291152 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:15.291163 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:15.291849 (+ 686us) server_negotiation.cc:407] Connection header received
0809 03:28:15.292128 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:15.292136 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:15.292364 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:15.292530 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:28:15.292971 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.294212 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.295616 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.296485 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.296614 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.296759 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:15.297085 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 03:28:15.297190 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":136,"threads_started":1}
I20260809 03:28:15.298770 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.291551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32840 (local address 127.9.46.194:39467)
0809 03:28:15.291947 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:15.291959 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:15.292395 (+ 436us) server_negotiation.cc:407] Connection header received
0809 03:28:15.292619 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:15.292625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:15.292809 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:15.292951 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:28:15.293178 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.294265 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.295671 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.296477 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.298310 (+ 1833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.298459 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:15.298513 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 03:28:15.298614 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":94,"threads_started":1}
I20260809 03:28:15.300384 26440 tablet_service.cc:1467] Processing CreateTablet for tablet 6d9640f7a27f4e8ba4f7152470cb57c5 (DEFAULT_TABLE table=test-table [id=527f0adcdfef4cfa9b25aeea645738fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:28:15.300925 26145 tablet_service.cc:1467] Processing CreateTablet for tablet 6d9640f7a27f4e8ba4f7152470cb57c5 (DEFAULT_TABLE table=test-table [id=527f0adcdfef4cfa9b25aeea645738fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:28:15.301498 26440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d9640f7a27f4e8ba4f7152470cb57c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:28:15.301326 26290 tablet_service.cc:1467] Processing CreateTablet for tablet 6d9640f7a27f4e8ba4f7152470cb57c5 (DEFAULT_TABLE table=test-table [id=527f0adcdfef4cfa9b25aeea645738fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:28:15.301934 26145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d9640f7a27f4e8ba4f7152470cb57c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:28:15.302248 26290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d9640f7a27f4e8ba4f7152470cb57c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:28:15.307788 26528 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: Bootstrap starting.
I20260809 03:28:15.309101 26530 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Bootstrap starting.
I20260809 03:28:15.309669 26529 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Bootstrap starting.
I20260809 03:28:15.310712 26528 tablet_bootstrap.cc:654] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: Neither blocks nor log segments found. Creating new log.
I20260809 03:28:15.311364 26529 tablet_bootstrap.cc:654] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Neither blocks nor log segments found. Creating new log.
I20260809 03:28:15.311606 26530 tablet_bootstrap.cc:654] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Neither blocks nor log segments found. Creating new log.
I20260809 03:28:15.311666 26528 log.cc:824] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: Log is configured to *not* fsync() on all Append() calls
I20260809 03:28:15.312141 26529 log.cc:824] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:28:15.312438 26530 log.cc:824] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Log is configured to *not* fsync() on all Append() calls
I20260809 03:28:15.314252 26529 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: No bootstrap required, opened a new log
I20260809 03:28:15.314257 26528 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: No bootstrap required, opened a new log
I20260809 03:28:15.314451 26528 ts_tablet_manager.cc:1397] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 03:28:15.314451 26529 ts_tablet_manager.cc:1397] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:28:15.315366 26530 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: No bootstrap required, opened a new log
I20260809 03:28:15.315572 26530 ts_tablet_manager.cc:1397] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 03:28:15.318406 26530 raft_consensus.cc:348] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:15.318441 26528 raft_consensus.cc:348] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:15.318701 26528 raft_consensus.cc:374] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:28:15.318702 26530 raft_consensus.cc:374] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:28:15.318771 26528 raft_consensus.cc:729] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713e900cfbfe47f0ab21680d78a1d418, State: Initialized, Role: FOLLOWER
I20260809 03:28:15.318770 26530 raft_consensus.cc:729] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f530e8fdc9d4798bd5d5457642946dc, State: Initialized, Role: FOLLOWER
I20260809 03:28:15.318971 26529 raft_consensus.cc:348] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:15.319202 26529 raft_consensus.cc:374] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:28:15.319262 26529 raft_consensus.cc:729] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b657f6beadf4ddba6409b756092968f, State: Initialized, Role: FOLLOWER
I20260809 03:28:15.319213 26530 consensus_queue.cc:260] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:15.319218 26528 consensus_queue.cc:260] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:15.319689 26529 consensus_queue.cc:260] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:15.320245 26528 ts_tablet_manager.cc:1428] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 03:28:15.320325 26530 ts_tablet_manager.cc:1428] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:28:15.320499 26529 ts_tablet_manager.cc:1428] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:28:15.320617 26511 heartbeater.cc:503] Master 127.9.46.254:46637 was elected leader, sending a full tablet report...
I20260809 03:28:15.320695 26363 heartbeater.cc:503] Master 127.9.46.254:46637 was elected leader, sending a full tablet report...
I20260809 03:28:15.320813 26217 heartbeater.cc:503] Master 127.9.46.254:46637 was elected leader, sending a full tablet report...
I20260809 03:28:15.433799 26534 raft_consensus.cc:482] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:28:15.434003 26534 raft_consensus.cc:504] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:15.435297 26534 leader_election.cc:288] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b657f6beadf4ddba6409b756092968f (127.9.46.193:44377), 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467)
I20260809 03:28:15.439165 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.435518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44377 (local address 127.9.46.195:36625)
0809 03:28:15.435678 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:15.435697 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:15.435800 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:15.436084 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:15.436089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:15.436112 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:28:15.436260 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.436267 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.437250 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.437254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.438049 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.438058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.438723 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.438727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.438834 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.438860 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 03:28:15.438949 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 03:28:15.439029 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 03:28:15.439164 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.435641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:36625 (local address 127.9.46.193:44377)
0809 03:28:15.435794 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:15.435801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:15.435822 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:28:15.435887 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:15.435890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:15.435952 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:15.436045 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:28:15.436383 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.437130 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.438177 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.438601 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.438729 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.438780 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:15.438968 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 03:28:15.439041 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:28:15.439371 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.435856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:46411 (local address 127.9.46.194:39467)
0809 03:28:15.436045 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:15.436049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:15.436186 (+ 137us) server_negotiation.cc:407] Connection header received
0809 03:28:15.436264 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:15.436266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:15.436307 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:15.436412 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:28:15.436838 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.437365 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.438408 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.438815 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.438945 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.438997 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:15.439191 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 03:28:15.439266 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 03:28:15.439368 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.435622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:39467 (local address 127.9.46.195:46411)
0809 03:28:15.436084 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:15.436097 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:15.436182 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:15.436428 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:15.436430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:15.436445 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:28:15.436534 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.436538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.437488 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.437493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.438272 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:15.438281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.438937 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.438940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:15.439057 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.439075 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 03:28:15.439160 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 03:28:15.439231 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":143,"threads_started":1}
I20260809 03:28:15.440167 26167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d9640f7a27f4e8ba4f7152470cb57c5" candidate_uuid: "713e900cfbfe47f0ab21680d78a1d418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b657f6beadf4ddba6409b756092968f" is_pre_election: true
I20260809 03:28:15.440153 26311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d9640f7a27f4e8ba4f7152470cb57c5" candidate_uuid: "713e900cfbfe47f0ab21680d78a1d418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f530e8fdc9d4798bd5d5457642946dc" is_pre_election: true
I20260809 03:28:15.440485 26311 raft_consensus.cc:2454] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 713e900cfbfe47f0ab21680d78a1d418 in term 0.
I20260809 03:28:15.440485 26167 raft_consensus.cc:2454] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 713e900cfbfe47f0ab21680d78a1d418 in term 0.
I20260809 03:28:15.441093 26390 leader_election.cc:302] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 713e900cfbfe47f0ab21680d78a1d418, 7b657f6beadf4ddba6409b756092968f, 7f530e8fdc9d4798bd5d5457642946dc; no voters:
I20260809 03:28:15.441344 26534 raft_consensus.cc:2781] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:28:15.441462 26534 raft_consensus.cc:482] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:28:15.441524 26534 raft_consensus.cc:3037] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:28:15.442821 26534 raft_consensus.cc:504] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:15.443224 26534 leader_election.cc:288] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [CANDIDATE]: Term 1 election: Requested vote from peers 7b657f6beadf4ddba6409b756092968f (127.9.46.193:44377), 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467)
I20260809 03:28:15.443574 26167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d9640f7a27f4e8ba4f7152470cb57c5" candidate_uuid: "713e900cfbfe47f0ab21680d78a1d418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b657f6beadf4ddba6409b756092968f"
I20260809 03:28:15.443639 26311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d9640f7a27f4e8ba4f7152470cb57c5" candidate_uuid: "713e900cfbfe47f0ab21680d78a1d418" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f530e8fdc9d4798bd5d5457642946dc"
I20260809 03:28:15.443713 26167 raft_consensus.cc:3037] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:28:15.443753 26311 raft_consensus.cc:3037] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:28:15.445116 26311 raft_consensus.cc:2454] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 713e900cfbfe47f0ab21680d78a1d418 in term 1.
I20260809 03:28:15.445116 26167 raft_consensus.cc:2454] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 713e900cfbfe47f0ab21680d78a1d418 in term 1.
I20260809 03:28:15.445433 26389 leader_election.cc:302] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 713e900cfbfe47f0ab21680d78a1d418, 7b657f6beadf4ddba6409b756092968f; no voters:
I20260809 03:28:15.445598 26534 raft_consensus.cc:2781] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:28:15.445828 26534 raft_consensus.cc:686] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 1 LEADER]: Becoming Leader. State: Replica: 713e900cfbfe47f0ab21680d78a1d418, State: Running, Role: LEADER
I20260809 03:28:15.446169 26534 consensus_queue.cc:237] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:15.449417 26006 catalog_manager.cc:5495] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 reported cstate change: term changed from 0 to 1, leader changed from <none> to 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "713e900cfbfe47f0ab21680d78a1d418" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } health_report { overall_health: UNKNOWN } } }
I20260809 03:28:15.485925 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.482495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37448 (local address 127.9.46.195:41861)
0809 03:28:15.482651 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:15.482654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:15.482668 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:28:15.482760 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:15.482762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:15.482818 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:15.482914 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:28:15.483143 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.483638 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.484449 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:15.484732 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:15.484817 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:15.485095 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:28:15.485542 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:28:15.485637 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:15.485744 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 03:28:15.485797 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 03:28:15.491888 26167 raft_consensus.cc:1261] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Refusing update from remote peer 713e900cfbfe47f0ab21680d78a1d418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:28:15.491888 26311 raft_consensus.cc:1261] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 FOLLOWER]: Refusing update from remote peer 713e900cfbfe47f0ab21680d78a1d418: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:28:15.492671 26534 consensus_queue.cc:1035] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:28:15.492883 26538 consensus_queue.cc:1035] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 03:28:15.496318 26512 tablet.cc:2369] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:28:15.500967 26543 mvcc.cc:205] Tried to move back new op lower bound from 7316464007126851584 to 7316464006950309888. Current Snapshot: MvccSnapshot[applied={T|T < 7316464007126851584}]
I20260809 03:28:15.501919 26542 mvcc.cc:205] Tried to move back new op lower bound from 7316464007126851584 to 7316464006950309888. Current Snapshot: MvccSnapshot[applied={T|T < 7316464007126851584}]
I20260809 03:28:15.502404 26541 mvcc.cc:205] Tried to move back new op lower bound from 7316464007126851584 to 7316464006950309888. Current Snapshot: MvccSnapshot[applied={T|T < 7316464007126851584 or (T in {7316464007126851584})}]
I20260809 03:28:15.508961 26007 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:28:15.509138 26007 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:28:15.511381 26007 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Sending DeleteTablet for 3 replicas of tablet 6d9640f7a27f4e8ba4f7152470cb57c5
I20260809 03:28:15.512269 26290 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d9640f7a27f4e8ba4f7152470cb57c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:28:15 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:32840
I20260809 03:28:15.512313 26440 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d9640f7a27f4e8ba4f7152470cb57c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:28:15 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:43618
I20260809 03:28:15.512336 26145 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d9640f7a27f4e8ba4f7152470cb57c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:28:15 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:43582
I20260809 03:28:15.512804 26556 tablet_replica.cc:330] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: stopping tablet replica
I20260809 03:28:15.512830 26555 tablet_replica.cc:330] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: stopping tablet replica
I20260809 03:28:15.512982 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 26077
I20260809 03:28:15.516784 26556 raft_consensus.cc:2229] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:28:15.517234 26556 raft_consensus.cc:2258] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:28:15.521235 26556 ts_tablet_manager.cc:1905] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:28:15.523995 26556 ts_tablet_manager.cc:1918] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:28:15.524104 26556 log.cc:1196] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/6d9640f7a27f4e8ba4f7152470cb57c5
I20260809 03:28:15.524438 26556 ts_tablet_manager.cc:1939] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 713e900cfbfe47f0ab21680d78a1d418: Deleting consensus metadata
W20260809 03:28:15.525466 25993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:44377 (error 108)
I20260809 03:28:15.525667 25994 catalog_manager.cc:4841] TS 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861): tablet 6d9640f7a27f4e8ba4f7152470cb57c5 (table test-table [id=527f0adcdfef4cfa9b25aeea645738fd]) successfully deleted
I20260809 03:28:15.526415 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 26222
I20260809 03:28:15.534651 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.534336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44377 (local address 127.0.0.1:47264)
0809 03:28:15.534488 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:15.534565 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 03:28:15.534879 25993 catalog_manager.cc:4584] TS 7b657f6beadf4ddba6409b756092968f (127.9.46.193:44377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d9640f7a27f4e8ba4f7152470cb57c5: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44377: connect: Connection refused (error 111)
I20260809 03:28:15.540940 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:15.540678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:39467 (local address 127.0.0.1:38568)
0809 03:28:15.540806 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:15.540865 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.194:39467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 03:28:15.541067 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 26369
I20260809 03:28:15.550866 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 25972
I20260809 03:28:35.558992 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46637
--webserver_interface=127.9.46.254
--webserver_port=40929
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:46637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:28:35.672884 26561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:35.673142 26561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:35.673182 26561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:35.677747 26561 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 03:28:35.677798 26561 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 03:28:35.677814 26561 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 03:28:35.677829 26561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:28:35.677846 26561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:35.677865 26561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:35.677881 26561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:28:35.677910 26561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:28:35.683176 26561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:46637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=40929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:35.684151 26561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:35.685180 26561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:35.698761 26570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:35.698813 26568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:35.698813 26567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:35.698891 26561 server_base.cc:1034] running on GCE node
I20260809 03:28:35.699651 26561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:35.700831 26561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:35.702001 26561 hybrid_clock.cc:648] HybridClock initialized: now 1786246115701973 us; error 40 us; skew 500 ppm
Aug 09 03:28:35 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246115, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:35.704993 26561 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:28:35.705914 26561 webserver.cc:460] Webserver started at http://127.9.46.254:40929/ using document root <none> and password file <none>
I20260809 03:28:35.706456 26561 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:35.706527 26561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:35.710397 26561 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:28:35.712761 26577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:35.714208 26561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:28:35.714360 26561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "ef06b268ae674451842bef38cfdd3f1d"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "5a604c11b0ceaf364aa244f32728b419"
server_key_iv: "13f26adad7302893de87812fab0d72f8"
server_key_version: "encryptionkey@0"
I20260809 03:28:35.714984 26561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:35.737015 26561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:35.740046 26561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:35.740272 26561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:35.747826 26561 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:46637
I20260809 03:28:35.748143 26631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:46637 every 8 connection(s)
I20260809 03:28:35.748981 26561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:28:35.753368 26632 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:28:35.754990 26632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Bootstrap starting.
I20260809 03:28:35.756804 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 26561
I20260809 03:28:35.757279 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:44377
--local_ip_for_outbound_sockets=127.9.46.193
--tserver_master_addrs=127.9.46.254:46637
--webserver_port=45709
--webserver_interface=127.9.46.193
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:28:35.778388 26632 log.cc:824] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:28:35.789397 26632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:28:35.789829 26632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Bootstrap complete.
I20260809 03:28:35.795511 26632 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:35.796022 26632 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef06b268ae674451842bef38cfdd3f1d, State: Initialized, Role: FOLLOWER
I20260809 03:28:35.796597 26632 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:35.796746 26632 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:28:35.796821 26632 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:28:35.796917 26632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 1 FOLLOWER]: Advancing to term 2
I20260809 03:28:35.799424 26632 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:35.799799 26632 leader_election.cc:302] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef06b268ae674451842bef38cfdd3f1d; no voters:
I20260809 03:28:35.800172 26632 leader_election.cc:288] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 03:28:35.800329 26636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 2 FOLLOWER]: Leader election won for term 2
I20260809 03:28:35.800680 26636 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 2 LEADER]: Becoming Leader. State: Replica: ef06b268ae674451842bef38cfdd3f1d, State: Running, Role: LEADER
I20260809 03:28:35.801050 26636 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:35.801604 26632 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:28:35.802502 26638 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef06b268ae674451842bef38cfdd3f1d. Latest consensus state: current_term: 2 leader_uuid: "ef06b268ae674451842bef38cfdd3f1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } } }
I20260809 03:28:35.802652 26638 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: This master's current role is: LEADER
I20260809 03:28:35.803375 26637 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ef06b268ae674451842bef38cfdd3f1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } } }
I20260809 03:28:35.803484 26637 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: This master's current role is: LEADER
I20260809 03:28:35.806681 26654 catalog_manager.cc:1259] Loaded cluster ID: 89ffc2cb4d444f90984b48868228dbcf
I20260809 03:28:35.806766 26654 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: loading cluster ID for follower catalog manager: success
I20260809 03:28:35.808760 26654 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: acquiring CA information for follower catalog manager: success
I20260809 03:28:35.810025 26654 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:28:35.810185 26646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:28:35.812315 26646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:28:35.812870 26646 catalog_manager.cc:1259] Loaded cluster ID: 89ffc2cb4d444f90984b48868228dbcf
I20260809 03:28:35.812952 26646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:28:35.813654 26646 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:28:35.814119 26646 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Loaded TSK: 0
I20260809 03:28:35.826637 26646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Generated new TSK 1
I20260809 03:28:35.826910 26646 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:28:35.892226 26634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:35.892506 26634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:35.892638 26634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:35.896736 26634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:35.896809 26634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:35.896896 26634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:28:35.902230 26634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:44377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=45709
--tserver_master_addrs=127.9.46.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:35.903293 26634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:35.904278 26634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:35.914772 26661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:35.914947 26634 server_base.cc:1034] running on GCE node
W20260809 03:28:35.914813 26664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:35.915089 26662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:35.915524 26634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:35.916235 26634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:35.917416 26634 hybrid_clock.cc:648] HybridClock initialized: now 1786246115917398 us; error 35 us; skew 500 ppm
Aug 09 03:28:35 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246115, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:35.920403 26634 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:28:35.921339 26634 webserver.cc:460] Webserver started at http://127.9.46.193:45709/ using document root <none> and password file <none>
I20260809 03:28:35.921900 26634 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:35.921977 26634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:35.926023 26634 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 03:28:35.928309 26671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:35.929808 26634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:28:35.929946 26634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "7b657f6beadf4ddba6409b756092968f"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "378a4c859c1bedd0854191c9f52435d8"
server_key_iv: "f93ece1d262adc6b212539a433b7efc2"
server_key_version: "encryptionkey@0"
I20260809 03:28:35.930620 26634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:35.947641 26634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:35.951046 26634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:35.951274 26634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:35.951973 26634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:28:35.953179 26681 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:28:35.957403 26634 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:28:35.957469 26634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260809 03:28:35.957518 26634 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:28:35.959236 26634 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:28:35.959295 26634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 03:28:35.959545 26681 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Bootstrap starting.
I20260809 03:28:35.970857 26634 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:44377
I20260809 03:28:35.970953 26796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:44377 every 8 connection(s)
I20260809 03:28:35.972064 26634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:28:35.976018 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 26634
I20260809 03:28:35.976480 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:39467
--local_ip_for_outbound_sockets=127.9.46.194
--tserver_master_addrs=127.9.46.254:46637
--webserver_port=44219
--webserver_interface=127.9.46.194
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:28:35.982472 26681 log.cc:824] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:28:35 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246115, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:28:35.989107 26799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:35.974274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:45407 (local address 127.9.46.254:46637)
0809 03:28:35.974633 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:35.974653 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:35.974971 (+ 318us) server_negotiation.cc:407] Connection header received
0809 03:28:35.976135 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:35.976150 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:35.976639 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:35.977068 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:35.978576 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:35.979557 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:35.980761 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:35.981220 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:35.981411 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:35.984152 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:35.984157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:35.984168 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:35.984194 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:35.985930 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:35.986479 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:35.986484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:35.986488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:35.986611 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:35.986961 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:35.986965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:35.986966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:35.987301 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:35.987448 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:35.987962 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 03:28:35.988218 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":122,"threads_started":1}
I20260809 03:28:35.989127 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:35.974303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:45407)
0809 03:28:35.974803 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:35.974844 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:35.975633 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:35.977142 (+ 1509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:35.977153 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:35.977659 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:28:35.978353 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:35.978378 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:35.979721 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:35.979724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:35.980638 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:35.980648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:35.981377 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:35.981381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:35.981584 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:35.982198 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:28:35.982214 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:28:35.983942 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:28:35.986069 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:35.986075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:35.986087 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:35.986350 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:35.986715 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:35.986719 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:35.986721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:35.986862 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:35.987420 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:28:35.987426 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:28:35.987667 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 03:28:35.987847 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 03:28:35.988193 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":134,"threads_started":1}
I20260809 03:28:35.991648 26797 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46637
I20260809 03:28:35.991882 26797 heartbeater.cc:464] Registering TS with master...
I20260809 03:28:35.992178 26681 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:28:35.992408 26797 heartbeater.cc:511] Master 127.9.46.254:46637 requested a full tablet report, sending...
I20260809 03:28:35.992588 26681 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Bootstrap complete.
I20260809 03:28:35.994174 26681 ts_tablet_manager.cc:1397] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Time spent bootstrapping tablet: real 0.035s user 0.018s sys 0.015s
I20260809 03:28:35.994755 26595 ts_manager.cc:194] Registered new tserver with Master: 7b657f6beadf4ddba6409b756092968f (127.9.46.193:44377)
I20260809 03:28:35.996981 26681 raft_consensus.cc:348] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:35.997586 26681 raft_consensus.cc:729] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b657f6beadf4ddba6409b756092968f, State: Initialized, Role: FOLLOWER
I20260809 03:28:35.998135 26595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:45407
I20260809 03:28:35.998152 26681 consensus_queue.cc:260] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:35.999275 26681 ts_tablet_manager.cc:1428] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 03:28:36.000741 26797 heartbeater.cc:503] Master 127.9.46.254:46637 was elected leader, sending a full tablet report...
Aug 09 03:28:36 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246115, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.193@KRBTEST.COM
I20260809 03:28:36.008836 26804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:35.998122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33736 (local address 127.9.46.193:44377)
0809 03:28:35.999247 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:35.999258 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:35.999279 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:28:35.999337 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:35.999342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:35.999518 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:35.999652 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:36.000383 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.001440 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.002533 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.002941 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.003068 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.005331 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:36.005341 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:36.005353 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:36.005381 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:36.006916 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.007402 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.007408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.007410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.007498 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.007789 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.007794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.007796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.008014 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:36.008163 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:36.008494 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 03:28:36.008665 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1040,"thread_start_us":101,"threads_started":1}
I20260809 03:28:36.008836 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:35.997411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44377 (local address 127.0.0.1:33736)
0809 03:28:35.998571 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:35.998612 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:35.998732 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:35.999688 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:35.999694 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:36.000117 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:28:36.000250 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.000257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.001541 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.001546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.002396 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.002405 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.003062 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.003066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.003181 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.003768 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:28:36.003780 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:28:36.005193 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:28:36.007031 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:36.007043 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:36.007048 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:36.007281 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:36.007599 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:36.007602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:36.007604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:36.007697 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:36.008139 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:28:36.008145 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:28:36.008274 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 03:28:36.008413 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 03:28:36.008659 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":119,"threads_started":1}
I20260809 03:28:36.009845 26728 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d9640f7a27f4e8ba4f7152470cb57c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:28:15 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:33736
I20260809 03:28:36.010632 26806 tablet_replica.cc:330] stopping tablet replica
I20260809 03:28:36.010913 26806 raft_consensus.cc:2229] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:28:36.011169 26806 raft_consensus.cc:2258] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:28:36.012079 26806 ts_tablet_manager.cc:1905] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:28:36.015650 26806 ts_tablet_manager.cc:1918] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:28:36.015731 26806 log.cc:1196] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal/wals/6d9640f7a27f4e8ba4f7152470cb57c5
I20260809 03:28:36.015988 26806 ts_tablet_manager.cc:1939] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7b657f6beadf4ddba6409b756092968f: Deleting consensus metadata
I20260809 03:28:36.017623 26580 catalog_manager.cc:4841] TS 7b657f6beadf4ddba6409b756092968f (127.9.46.193:44377): tablet 6d9640f7a27f4e8ba4f7152470cb57c5 (table test-table [id=527f0adcdfef4cfa9b25aeea645738fd]) successfully deleted
W20260809 03:28:36.095923 26801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:36.096194 26801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:36.096268 26801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:36.100206 26801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:36.100294 26801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:36.100382 26801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:28:36.105767 26801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:39467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=44219
--tserver_master_addrs=127.9.46.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:36.106817 26801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:36.107827 26801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:36.117403 26811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:36.117400 26814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:36.117429 26812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:36.117643 26801 server_base.cc:1034] running on GCE node
I20260809 03:28:36.118151 26801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:36.118813 26801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:36.119994 26801 hybrid_clock.cc:648] HybridClock initialized: now 1786246116119976 us; error 29 us; skew 500 ppm
Aug 09 03:28:36 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:36.122624 26801 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:28:36.123370 26801 webserver.cc:460] Webserver started at http://127.9.46.194:44219/ using document root <none> and password file <none>
I20260809 03:28:36.123873 26801 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:36.123939 26801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:36.127534 26801 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:28:36.129529 26821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:36.130682 26801 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:28:36.130775 26801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "7f530e8fdc9d4798bd5d5457642946dc"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "caa211c7dfad659c1809550acc1e1e65"
server_key_iv: "0baba4b889b14d55121bedc6c0460b3a"
server_key_version: "encryptionkey@0"
I20260809 03:28:36.131261 26801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:36.145884 26801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:36.148954 26801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:36.149109 26801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:36.149633 26801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:28:36.150607 26828 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:28:36.154546 26801 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:28:36.154609 26801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260809 03:28:36.154687 26801 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:28:36.156296 26801 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:28:36.156342 26801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 03:28:36.156664 26828 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Bootstrap starting.
I20260809 03:28:36.167052 26801 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:39467
I20260809 03:28:36.167114 26947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:39467 every 8 connection(s)
I20260809 03:28:36.168061 26801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:28:36.175554 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 26801
I20260809 03:28:36.176101 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:41861
--local_ip_for_outbound_sockets=127.9.46.195
--tserver_master_addrs=127.9.46.254:46637
--webserver_port=35605
--webserver_interface=127.9.46.195
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:28:36.176280 26828 log.cc:824] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:28:36 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:28:36.183139 26799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.169941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:43207 (local address 127.9.46.254:46637)
0809 03:28:36.170107 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:36.170112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:36.170699 (+ 587us) server_negotiation.cc:407] Connection header received
0809 03:28:36.171690 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:36.171693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:36.171750 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:36.171857 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:36.173038 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.173954 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.175097 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.175407 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.175543 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.178697 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:36.178704 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:36.178707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:36.178742 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:36.180618 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.181226 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.181233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.181238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.181341 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.181751 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.181758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.181763 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.182039 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:36.182158 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:36.182721 (+ 563us) server_negotiation.cc:299] Negotiation successful
0809 03:28:36.182885 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 03:28:36.183845 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.170061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:43207)
0809 03:28:36.170527 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:36.170579 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:36.171482 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:36.171995 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:36.172001 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:36.172361 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:28:36.172885 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.172898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.174111 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.174114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.174957 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.174965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.175565 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.175570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.175785 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.176522 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:28:36.176571 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:28:36.178501 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:28:36.180743 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:36.180748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:36.180768 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:36.181105 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:36.181449 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:36.181452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:36.181454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:36.181635 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:36.182165 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:28:36.182171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:28:36.182437 (+ 266us) client_negotiation.cc:769] Sending connection context
0809 03:28:36.182640 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 03:28:36.182939 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":144,"threads_started":1}
I20260809 03:28:36.184423 26828 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:28:36.185029 26828 tablet_bootstrap.cc:492] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Bootstrap complete.
I20260809 03:28:36.185194 26948 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46637
I20260809 03:28:36.185406 26948 heartbeater.cc:464] Registering TS with master...
I20260809 03:28:36.185935 26948 heartbeater.cc:511] Master 127.9.46.254:46637 requested a full tablet report, sending...
I20260809 03:28:36.186409 26828 ts_tablet_manager.cc:1397] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Time spent bootstrapping tablet: real 0.030s user 0.024s sys 0.004s
I20260809 03:28:36.187879 26595 ts_manager.cc:194] Registered new tserver with Master: 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467)
I20260809 03:28:36.189462 26595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:43207
I20260809 03:28:36.188884 26828 raft_consensus.cc:348] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:36.189757 26828 raft_consensus.cc:729] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f530e8fdc9d4798bd5d5457642946dc, State: Initialized, Role: FOLLOWER
I20260809 03:28:36.190299 26828 consensus_queue.cc:260] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:36.191332 26828 ts_tablet_manager.cc:1428] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:28:36.191918 26948 heartbeater.cc:503] Master 127.9.46.254:46637 was elected leader, sending a full tablet report...
Aug 09 03:28:36 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246115, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.194@KRBTEST.COM
I20260809 03:28:36.199589 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.189021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:39467 (local address 127.0.0.1:41492)
0809 03:28:36.189788 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:36.189802 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:36.189882 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:36.190701 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:36.190704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:36.190884 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:28:36.190983 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.190989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.192637 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.192640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.193462 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.193478 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.194197 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.194199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.194267 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.194767 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:28:36.194778 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:28:36.195960 (+ 1182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:28:36.198153 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:36.198155 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:36.198156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:36.198336 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:36.198643 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:36.198645 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:36.198646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:36.198719 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:36.199176 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:28:36.199178 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:28:36.199255 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 03:28:36.199351 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 03:28:36.199460 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702}
I20260809 03:28:36.199769 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.189269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41492 (local address 127.9.46.194:39467)
0809 03:28:36.189605 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:36.189618 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:36.190220 (+ 602us) server_negotiation.cc:407] Connection header received
0809 03:28:36.190283 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:36.190288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:36.190548 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:36.191124 (+ 576us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:36.191247 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.192530 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.193580 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.194083 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.194190 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.196103 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:36.196112 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:36.196126 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:36.196151 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:36.198029 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.198439 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.198445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.198447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.198533 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.198807 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.198810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.198812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.199066 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:36.199190 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:36.199454 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 03:28:36.199603 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":188,"spinlock_wait_cycles":4480,"thread_start_us":114,"threads_started":1}
I20260809 03:28:36.200793 26876 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d9640f7a27f4e8ba4f7152470cb57c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:28:15 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:41492
I20260809 03:28:36.201819 26955 tablet_replica.cc:330] stopping tablet replica
I20260809 03:28:36.202106 26955 raft_consensus.cc:2229] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:28:36.202343 26955 raft_consensus.cc:2258] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:28:36.203295 26955 ts_tablet_manager.cc:1905] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:28:36.206859 26955 ts_tablet_manager.cc:1918] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:28:36.206972 26955 log.cc:1196] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/6d9640f7a27f4e8ba4f7152470cb57c5
I20260809 03:28:36.207247 26955 ts_tablet_manager.cc:1939] T 6d9640f7a27f4e8ba4f7152470cb57c5 P 7f530e8fdc9d4798bd5d5457642946dc: Deleting consensus metadata
I20260809 03:28:36.208679 26581 catalog_manager.cc:4841] TS 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467): tablet 6d9640f7a27f4e8ba4f7152470cb57c5 (table test-table [id=527f0adcdfef4cfa9b25aeea645738fd]) successfully deleted
W20260809 03:28:36.292939 26952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:36.293215 26952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:36.293285 26952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:36.297389 26952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:36.297464 26952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:36.297546 26952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:28:36.302775 26952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:41861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=35605
--tserver_master_addrs=127.9.46.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:36.303865 26952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:36.304889 26952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:36.314244 26961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:36.314244 26960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:36.314460 26952 server_base.cc:1034] running on GCE node
W20260809 03:28:36.314265 26963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:36.315083 26952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:36.315671 26952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:36.316824 26952 hybrid_clock.cc:648] HybridClock initialized: now 1786246116316811 us; error 51 us; skew 500 ppm
Aug 09 03:28:36 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:36.319569 26952 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:28:36.320384 26952 webserver.cc:460] Webserver started at http://127.9.46.195:35605/ using document root <none> and password file <none>
I20260809 03:28:36.320994 26952 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:36.321072 26952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:36.324869 26952 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 03:28:36.326905 26970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:36.328126 26952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:28:36.328265 26952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "713e900cfbfe47f0ab21680d78a1d418"
format_stamp: "Formatted at 2026-08-09 03:28:15 on dist-test-slave-65mx"
server_key: "75425b83b82a486aefb2dfd2e8e4ac35"
server_key_iv: "34005c4bf8ecef2814b6ed9bd44093f9"
server_key_version: "encryptionkey@0"
I20260809 03:28:36.328857 26952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:36.344355 26952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:36.347132 26952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:36.347342 26952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:36.347924 26952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:28:36.348809 26952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:28:36.348879 26952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:36.348953 26952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:28:36.348995 26952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:36.357900 26952 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:41861
I20260809 03:28:36.357926 27094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:41861 every 8 connection(s)
I20260809 03:28:36.358852 26952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:28:36.364391 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 26952
Aug 09 03:28:36 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246116, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:28:36.372776 26799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.360531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:49301 (local address 127.9.46.254:46637)
0809 03:28:36.360697 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:36.360702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:36.361554 (+ 852us) server_negotiation.cc:407] Connection header received
0809 03:28:36.362752 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:36.362754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:36.362802 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:36.362878 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:36.363954 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.364456 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.365957 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.366173 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.366252 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.369116 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:36.369122 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:36.369129 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:36.369148 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:36.370721 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.371226 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.371229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.371231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.371315 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.371694 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.371697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.371699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.371871 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:36.371978 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:36.372457 (+ 479us) server_negotiation.cc:299] Negotiation successful
0809 03:28:36.372603 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:28:36.373641 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.365135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58564 (local address 127.9.46.254:46637)
0809 03:28:36.365368 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:36.365371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:36.365516 (+ 145us) server_negotiation.cc:407] Connection header received
0809 03:28:36.365661 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:36.365663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:36.365700 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:36.365769 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:36.366702 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.367385 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.368434 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.368812 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.368931 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.370251 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:36.370255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:36.370257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:36.370302 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:36.371918 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.372346 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.372350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.372352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.372436 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.372743 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.372746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.372748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.372997 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:36.373195 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:36.373386 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 03:28:36.373479 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":64,"threads_started":1}
I20260809 03:28:36.373796 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.360976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:49301)
0809 03:28:36.361406 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:36.361447 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:36.362572 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:36.363033 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:36.363039 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:36.363378 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:28:36.363830 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.363843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.364645 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.364648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.365842 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.365852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.366337 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.366341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.366548 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.367241 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:28:36.367257 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:28:36.368975 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:28:36.370835 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:36.370841 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:36.370854 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:36.371117 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:36.371409 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:36.371413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:36.371415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:36.371589 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:36.371976 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:28:36.371983 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:28:36.372220 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 03:28:36.372394 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 03:28:36.372733 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":101,"threads_started":1}
I20260809 03:28:36.374867 27095 heartbeater.cc:347] Connected to a master server at 127.9.46.254:46637
I20260809 03:28:36.375120 27095 heartbeater.cc:464] Registering TS with master...
I20260809 03:28:36.375617 27095 heartbeater.cc:511] Master 127.9.46.254:46637 requested a full tablet report, sending...
I20260809 03:28:36.376652 26595 ts_manager.cc:194] Registered new tserver with Master: 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861)
I20260809 03:28:36.377197 26595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:49301
I20260809 03:28:36.387013 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:28:36.390645 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.387601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.9.46.254:46637)
0809 03:28:36.387715 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:36.387718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:36.387902 (+ 184us) server_negotiation.cc:407] Connection header received
0809 03:28:36.388015 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:36.388017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:36.388107 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:36.388187 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:28:36.388367 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.388906 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.389700 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.389910 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.389983 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.390255 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:28:36.390367 (+ 112us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 03:28:36.390550 (+ 183us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58574: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 03:28:36.390722 27100 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58574: token expired
W20260809 03:28:36.391053 26516 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.46.254:46637)
I20260809 03:28:36.398260 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.391409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58584 (local address 127.9.46.254:46637)
0809 03:28:36.391518 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:36.391523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:36.391608 (+ 85us) server_negotiation.cc:407] Connection header received
0809 03:28:36.391736 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:36.391739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:36.391802 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:36.391882 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:36.392472 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.393012 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.393805 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.394028 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.394137 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.395373 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:36.395376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:36.395379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:36.395403 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:36.396615 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.397039 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.397042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.397044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.397136 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:36.397401 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:36.397405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:36.397407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:36.397581 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:36.397715 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:36.397941 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 03:28:36.398067 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:28:36.400432 26595 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:28:36.403807 26595 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:28:36.411226 26876 tablet_service.cc:1467] Processing CreateTablet for tablet 7c73f86cde8d4332b6fe19fbc04a5e77 (DEFAULT_TABLE table=test-table [id=6715de3aa399497a8bd4202934e2f4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:28:36.411485 26728 tablet_service.cc:1467] Processing CreateTablet for tablet 7c73f86cde8d4332b6fe19fbc04a5e77 (DEFAULT_TABLE table=test-table [id=6715de3aa399497a8bd4202934e2f4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:28:36.411821 26876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c73f86cde8d4332b6fe19fbc04a5e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:28:36.412024 26728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c73f86cde8d4332b6fe19fbc04a5e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:28:36.414598 26828 tablet_bootstrap.cc:492] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: Bootstrap starting.
I20260809 03:28:36.414830 26681 tablet_bootstrap.cc:492] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Bootstrap starting.
I20260809 03:28:36.415549 26681 tablet_bootstrap.cc:654] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Neither blocks nor log segments found. Creating new log.
I20260809 03:28:36.415968 26828 tablet_bootstrap.cc:654] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: Neither blocks nor log segments found. Creating new log.
I20260809 03:28:36.417207 26828 tablet_bootstrap.cc:492] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: No bootstrap required, opened a new log
I20260809 03:28:36.417310 26828 ts_tablet_manager.cc:1397] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 03:28:36.417414 26681 tablet_bootstrap.cc:492] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: No bootstrap required, opened a new log
I20260809 03:28:36.417476 26681 ts_tablet_manager.cc:1397] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 03:28:36.417722 26828 raft_consensus.cc:348] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:36.417834 26828 raft_consensus.cc:374] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:28:36.417798 26681 raft_consensus.cc:348] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:36.417876 26681 raft_consensus.cc:374] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:28:36.417870 26828 raft_consensus.cc:729] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f530e8fdc9d4798bd5d5457642946dc, State: Initialized, Role: FOLLOWER
I20260809 03:28:36.417902 26681 raft_consensus.cc:729] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b657f6beadf4ddba6409b756092968f, State: Initialized, Role: FOLLOWER
I20260809 03:28:36.417994 26828 consensus_queue.cc:260] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:36.418037 26681 consensus_queue.cc:260] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:36.418239 26828 ts_tablet_manager.cc:1428] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 03:28:36.418390 26681 ts_tablet_manager.cc:1428] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260809 03:28:36.419082 26949 tablet.cc:2369] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:28:36.419885 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.410391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:41861 (local address 127.0.0.1:46138)
0809 03:28:36.410505 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:36.410520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:36.410614 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:36.415647 (+ 5033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:36.415653 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:36.415675 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:28:36.415789 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.415797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.417304 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.417308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.418496 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.418510 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.419571 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.419574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.419642 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.419681 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 03:28:36.419730 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 03:28:36.419789 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 03:28:36.420010 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.410861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46138 (local address 127.9.46.195:41861)
0809 03:28:36.415079 (+ 4218us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:36.415096 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:36.415126 (+ 30us) server_negotiation.cc:407] Connection header received
0809 03:28:36.415214 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:36.415222 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:36.415464 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:36.415636 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:28:36.415968 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.417172 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.418648 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.419424 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.419552 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.419698 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:36.419761 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 03:28:36.419868 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4096,"thread_start_us":198,"threads_started":1}
I20260809 03:28:36.422818 27021 tablet_service.cc:1467] Processing CreateTablet for tablet 7c73f86cde8d4332b6fe19fbc04a5e77 (DEFAULT_TABLE table=test-table [id=6715de3aa399497a8bd4202934e2f4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:28:36.423733 27021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c73f86cde8d4332b6fe19fbc04a5e77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:28:36.430095 27109 tablet_bootstrap.cc:492] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: Bootstrap starting.
I20260809 03:28:36.432413 27109 tablet_bootstrap.cc:654] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: Neither blocks nor log segments found. Creating new log.
I20260809 03:28:36.433272 27109 log.cc:824] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: Log is configured to *not* fsync() on all Append() calls
I20260809 03:28:36.435160 27109 tablet_bootstrap.cc:492] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: No bootstrap required, opened a new log
I20260809 03:28:36.435359 27109 ts_tablet_manager.cc:1397] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:28:36.438351 27109 raft_consensus.cc:348] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:36.438597 27109 raft_consensus.cc:374] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:28:36.438668 27109 raft_consensus.cc:729] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713e900cfbfe47f0ab21680d78a1d418, State: Initialized, Role: FOLLOWER
I20260809 03:28:36.439049 27109 consensus_queue.cc:260] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:36.439684 27095 heartbeater.cc:503] Master 127.9.46.254:46637 was elected leader, sending a full tablet report...
I20260809 03:28:36.439980 27109 ts_tablet_manager.cc:1428] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:28:36.459681 26954 raft_consensus.cc:482] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:28:36.459874 26954 raft_consensus.cc:504] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:36.460951 26954 leader_election.cc:288] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861), 7b657f6beadf4ddba6409b756092968f (127.9.46.193:44377)
I20260809 03:28:36.464526 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.461322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:50725 (local address 127.9.46.195:41861)
0809 03:28:36.461432 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:36.461436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:36.461451 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:28:36.461594 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:36.461597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:36.461657 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:36.461744 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:28:36.462015 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.462509 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.463551 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.463952 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.464092 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.464142 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:36.464341 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 03:28:36.464408 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:28:36.464548 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.461147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:41861 (local address 127.9.46.194:50725)
0809 03:28:36.461338 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:36.461355 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:36.461455 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:36.461782 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:36.461784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:36.461800 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:28:36.461898 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.461903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.462636 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.462641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.463424 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.463433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.464079 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.464082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.464196 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.464232 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 03:28:36.464324 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 03:28:36.464397 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260809 03:28:36.465215 26804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.461330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:58797 (local address 127.9.46.193:44377)
0809 03:28:36.461479 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:36.461485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:36.461936 (+ 451us) server_negotiation.cc:407] Connection header received
0809 03:28:36.462079 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:36.462082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:36.462138 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:36.462239 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:28:36.462509 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.463025 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.464042 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.464489 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.464690 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.464851 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:36.465027 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 03:28:36.465123 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:28:36.465229 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.461147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44377 (local address 127.9.46.194:58797)
0809 03:28:36.461813 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:36.461829 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:36.461964 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:36.462251 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:36.462254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:36.462269 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:28:36.462394 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.462399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.463145 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.463150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.463908 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:36.463916 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.464795 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.464799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:36.464902 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.464923 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:28:36.465006 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 03:28:36.465083 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":316,"thread_start_us":117,"threads_started":1}
I20260809 03:28:36.465227 27044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c73f86cde8d4332b6fe19fbc04a5e77" candidate_uuid: "7f530e8fdc9d4798bd5d5457642946dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713e900cfbfe47f0ab21680d78a1d418" is_pre_election: true
I20260809 03:28:36.465535 27044 raft_consensus.cc:2454] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f530e8fdc9d4798bd5d5457642946dc in term 0.
I20260809 03:28:36.465886 26749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c73f86cde8d4332b6fe19fbc04a5e77" candidate_uuid: "7f530e8fdc9d4798bd5d5457642946dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b657f6beadf4ddba6409b756092968f" is_pre_election: true
I20260809 03:28:36.466010 26825 leader_election.cc:302] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 713e900cfbfe47f0ab21680d78a1d418, 7f530e8fdc9d4798bd5d5457642946dc; no voters:
I20260809 03:28:36.466199 26749 raft_consensus.cc:2454] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f530e8fdc9d4798bd5d5457642946dc in term 0.
I20260809 03:28:36.466297 26954 raft_consensus.cc:2781] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:28:36.466392 26954 raft_consensus.cc:482] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:28:36.466483 26954 raft_consensus.cc:3037] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:28:36.467622 26954 raft_consensus.cc:504] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:36.468000 26954 leader_election.cc:288] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [CANDIDATE]: Term 1 election: Requested vote from peers 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861), 7b657f6beadf4ddba6409b756092968f (127.9.46.193:44377)
I20260809 03:28:36.468348 27044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c73f86cde8d4332b6fe19fbc04a5e77" candidate_uuid: "7f530e8fdc9d4798bd5d5457642946dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713e900cfbfe47f0ab21680d78a1d418"
I20260809 03:28:36.468379 26749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c73f86cde8d4332b6fe19fbc04a5e77" candidate_uuid: "7f530e8fdc9d4798bd5d5457642946dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b657f6beadf4ddba6409b756092968f"
I20260809 03:28:36.468487 26749 raft_consensus.cc:3037] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:28:36.468475 27044 raft_consensus.cc:3037] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:28:36.469774 26749 raft_consensus.cc:2454] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f530e8fdc9d4798bd5d5457642946dc in term 1.
I20260809 03:28:36.469774 27044 raft_consensus.cc:2454] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f530e8fdc9d4798bd5d5457642946dc in term 1.
I20260809 03:28:36.470042 26825 leader_election.cc:302] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 713e900cfbfe47f0ab21680d78a1d418, 7f530e8fdc9d4798bd5d5457642946dc; no voters:
I20260809 03:28:36.470248 26954 raft_consensus.cc:2781] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:28:36.470435 26954 raft_consensus.cc:686] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 LEADER]: Becoming Leader. State: Replica: 7f530e8fdc9d4798bd5d5457642946dc, State: Running, Role: LEADER
I20260809 03:28:36.470686 26954 consensus_queue.cc:237] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
W20260809 03:28:36.473306 26798 tablet.cc:2369] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:28:36.473614 26594 catalog_manager.cc:5495] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194). New cstate: current_term: 1 leader_uuid: "7f530e8fdc9d4798bd5d5457642946dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } health_report { overall_health: HEALTHY } } }
I20260809 03:28:36.497705 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.494517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.9.46.194:39467)
0809 03:28:36.494659 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:36.494663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:36.494676 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:28:36.494750 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:36.494752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:36.494804 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:36.494892 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:28:36.495102 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.495613 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.496378 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:36.496684 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:36.496767 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:36.497004 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:28:36.497401 (+ 397us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:28:36.497485 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:36.497547 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 03:28:36.497601 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:28:36.501478 26749 raft_consensus.cc:1261] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Refusing update from remote peer 7f530e8fdc9d4798bd5d5457642946dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:28:36.501482 27044 raft_consensus.cc:1261] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418 [term 1 FOLLOWER]: Refusing update from remote peer 7f530e8fdc9d4798bd5d5457642946dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:28:36.502156 26954 consensus_queue.cc:1035] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:28:36.502339 27113 consensus_queue.cc:1035] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:28:36.508571 27117 mvcc.cc:205] Tried to move back new op lower bound from 7316464093183680512 to 7316464093066149888. Current Snapshot: MvccSnapshot[applied={T|T < 7316464093183680512}]
I20260809 03:28:36.510577 27116 mvcc.cc:205] Tried to move back new op lower bound from 7316464093183680512 to 7316464093066149888. Current Snapshot: MvccSnapshot[applied={T|T < 7316464093183680512 or (T in {7316464093183680512})}]
I20260809 03:28:36.510773 27118 mvcc.cc:205] Tried to move back new op lower bound from 7316464093183680512 to 7316464093066149888. Current Snapshot: MvccSnapshot[applied={T|T < 7316464093183680512}]
I20260809 03:28:36.516016 26595 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:28:36.516156 26595 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:28:36.518402 26595 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Sending DeleteTablet for 3 replicas of tablet 7c73f86cde8d4332b6fe19fbc04a5e77
I20260809 03:28:36.519025 26728 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c73f86cde8d4332b6fe19fbc04a5e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:28:36 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:33736
I20260809 03:28:36.519178 26876 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c73f86cde8d4332b6fe19fbc04a5e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:28:36 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:41492
I20260809 03:28:36.519388 26955 tablet_replica.cc:330] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: stopping tablet replica
I20260809 03:28:36.519449 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 26634
I20260809 03:28:36.519409 27021 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c73f86cde8d4332b6fe19fbc04a5e77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:28:36 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:46138
I20260809 03:28:36.519513 26955 raft_consensus.cc:2229] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:28:36.519791 26955 raft_consensus.cc:2258] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:28:36.522117 27128 tablet_replica.cc:330] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: stopping tablet replica
I20260809 03:28:36.522428 27128 raft_consensus.cc:2229] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:28:36.522714 27128 raft_consensus.cc:2258] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:28:36.522800 26955 ts_tablet_manager.cc:1905] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:28:36.524345 27128 ts_tablet_manager.cc:1905] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:28:36.525379 26955 ts_tablet_manager.cc:1918] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:28:36.525449 26955 log.cc:1196] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/7c73f86cde8d4332b6fe19fbc04a5e77
I20260809 03:28:36.525722 26955 ts_tablet_manager.cc:1939] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7f530e8fdc9d4798bd5d5457642946dc: Deleting consensus metadata
I20260809 03:28:36.527182 27128 ts_tablet_manager.cc:1918] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:28:36.527282 27128 log.cc:1196] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/7c73f86cde8d4332b6fe19fbc04a5e77
I20260809 03:28:36.527637 27128 ts_tablet_manager.cc:1939] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 713e900cfbfe47f0ab21680d78a1d418: Deleting consensus metadata
I20260809 03:28:36.532513 26581 catalog_manager.cc:4841] TS 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861): tablet 7c73f86cde8d4332b6fe19fbc04a5e77 (table test-table [id=6715de3aa399497a8bd4202934e2f4fd]) successfully deleted
I20260809 03:28:36.532765 26581 catalog_manager.cc:4841] TS 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467): tablet 7c73f86cde8d4332b6fe19fbc04a5e77 (table test-table [id=6715de3aa399497a8bd4202934e2f4fd]) successfully deleted
W20260809 03:28:36.537930 26580 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:44377 (error 108)
I20260809 03:28:36.538527 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 26801
I20260809 03:28:36.539716 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:36.538731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44377 (local address 127.0.0.1:33752)
0809 03:28:36.539496 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:36.539597 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":654}
W20260809 03:28:36.542451 26580 catalog_manager.cc:4584] TS 7b657f6beadf4ddba6409b756092968f (127.9.46.193:44377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c73f86cde8d4332b6fe19fbc04a5e77: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44377: connect: Connection refused (error 111)
I20260809 03:28:36.547247 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 26952
I20260809 03:28:36.554184 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 26561
I20260809 03:28:56.562835 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46637
--webserver_interface=127.9.46.254
--webserver_port=40929
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:46637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:28:56.676204 27130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:56.676476 27130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:56.676518 27130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:56.681231 27130 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 03:28:56.681285 27130 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 03:28:56.681301 27130 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 03:28:56.681315 27130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:28:56.681357 27130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:56.681389 27130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:56.681422 27130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:28:56.681454 27130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:28:56.686823 27130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:46637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=40929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:56.687877 27130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:56.688983 27130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:56.697535 27138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:56.697592 27135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:56.697592 27136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:56.697644 27130 server_base.cc:1034] running on GCE node
I20260809 03:28:56.698457 27130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:56.699508 27130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:56.700623 27130 hybrid_clock.cc:648] HybridClock initialized: now 1786246136700591 us; error 38 us; skew 500 ppm
Aug 09 03:28:56 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246136, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:56.703353 27130 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:28:56.704245 27130 webserver.cc:460] Webserver started at http://127.9.46.254:40929/ using document root <none> and password file <none>
I20260809 03:28:56.704859 27130 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:56.704952 27130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:56.708652 27130 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 03:28:56.710654 27145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:56.711853 27130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:28:56.711994 27130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "ef06b268ae674451842bef38cfdd3f1d"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "5a604c11b0ceaf364aa244f32728b419"
server_key_iv: "13f26adad7302893de87812fab0d72f8"
server_key_version: "encryptionkey@0"
I20260809 03:28:56.712544 27130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:56.733575 27130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:56.736615 27130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:56.736840 27130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:56.744179 27130 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:46637
I20260809 03:28:56.744596 27205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:46637 every 8 connection(s)
I20260809 03:28:56.745357 27130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:28:56.749840 27208 sys_catalog.cc:263] Verifying existing consensus state
I20260809 03:28:56.750370 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 27130
I20260809 03:28:56.750905 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:44377
--local_ip_for_outbound_sockets=127.9.46.193
--tserver_master_addrs=127.9.46.254:46637
--webserver_port=45709
--webserver_interface=127.9.46.193
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:28:56.752183 27208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Bootstrap starting.
I20260809 03:28:56.770339 27208 log.cc:824] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Log is configured to *not* fsync() on all Append() calls
I20260809 03:28:56.781378 27208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:28:56.781775 27208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Bootstrap complete.
I20260809 03:28:56.785884 27208 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:56.786262 27208 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef06b268ae674451842bef38cfdd3f1d, State: Initialized, Role: FOLLOWER
I20260809 03:28:56.786769 27208 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:56.786881 27208 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:28:56.786964 27208 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:28:56.787084 27208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 2 FOLLOWER]: Advancing to term 3
I20260809 03:28:56.788959 27208 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:56.789283 27208 leader_election.cc:302] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef06b268ae674451842bef38cfdd3f1d; no voters:
I20260809 03:28:56.789618 27208 leader_election.cc:288] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 03:28:56.789705 27212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 3 FOLLOWER]: Leader election won for term 3
I20260809 03:28:56.790056 27212 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [term 3 LEADER]: Becoming Leader. State: Replica: ef06b268ae674451842bef38cfdd3f1d, State: Running, Role: LEADER
I20260809 03:28:56.790403 27212 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } }
I20260809 03:28:56.790853 27208 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:28:56.791730 27213 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ef06b268ae674451842bef38cfdd3f1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } } }
I20260809 03:28:56.791867 27213 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: This master's current role is: LEADER
I20260809 03:28:56.792229 27214 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef06b268ae674451842bef38cfdd3f1d. Latest consensus state: current_term: 3 leader_uuid: "ef06b268ae674451842bef38cfdd3f1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef06b268ae674451842bef38cfdd3f1d" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46637 } } }
I20260809 03:28:56.792376 27214 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d [sys.catalog]: This master's current role is: LEADER
I20260809 03:28:56.792829 27223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:28:56.798173 27223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:28:56.798813 27223 catalog_manager.cc:1259] Loaded cluster ID: 89ffc2cb4d444f90984b48868228dbcf
I20260809 03:28:56.798878 27223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:28:56.802037 27223 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:28:56.802757 27223 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Loaded TSK: 0
I20260809 03:28:56.802840 27223 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ef06b268ae674451842bef38cfdd3f1d: Loaded TSK: 1
F20260809 03:28:56.803510 27223 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fba4acad9ad google::LogMessage::Fail() at ??:0
@ 0x7fba4acb088a google::LogMessage::SendToLog() at ??:0
@ 0x7fba4acad4de google::LogMessage::Flush() at ??:0
@ 0x7fba4acaea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fba4d97118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fba4d969f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fba556167bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fba555f099c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fba555f1178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fba55632df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fba5565182b std::function<>::operator()() at ??:0
@ 0x7fba555f0b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fba555f1f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fba555ee579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fba55632640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fba4ceff3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fba4ceffc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fba4cf013f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fba4cef0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fba4e5346db start_thread at ??:0
@ 0x7fba497ab61f clone at ??:0
W20260809 03:28:56.893383 27210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:56.893697 27210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:56.893743 27210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:56.898231 27210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:56.898275 27210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:56.898337 27210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:28:56.904057 27210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:44377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=45709
--tserver_master_addrs=127.9.46.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:56.905146 27210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:56.906298 27210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:56.916496 27261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:56.916522 27260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:56.916657 27210 server_base.cc:1034] running on GCE node
W20260809 03:28:56.916491 27263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:56.917320 27210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:56.918049 27210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:56.919234 27210 hybrid_clock.cc:648] HybridClock initialized: now 1786246136919205 us; error 47 us; skew 500 ppm
Aug 09 03:28:56 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246136, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:56.922204 27210 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:28:56.923071 27210 webserver.cc:460] Webserver started at http://127.9.46.193:45709/ using document root <none> and password file <none>
I20260809 03:28:56.923733 27210 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:56.923830 27210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:56.928349 27210 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:28:56.930929 27270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:56.932302 27210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:28:56.932439 27210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "7b657f6beadf4ddba6409b756092968f"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "378a4c859c1bedd0854191c9f52435d8"
server_key_iv: "f93ece1d262adc6b212539a433b7efc2"
server_key_version: "encryptionkey@0"
I20260809 03:28:56.933061 27210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:56.953420 27210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:56.956843 27210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:56.957091 27210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:56.957731 27210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:28:56.958956 27277 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 03:28:56.964516 27210 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 03:28:56.964666 27210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 03:28:56.964756 27210 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 03:28:56.966475 27210 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 03:28:56.966544 27210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 03:28:56.966835 27277 tablet_bootstrap.cc:492] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Bootstrap starting.
I20260809 03:28:56.977533 27210 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:44377
I20260809 03:28:56.977794 27396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:44377 every 8 connection(s)
I20260809 03:28:56.978582 27210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:28:56.979104 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 27210
I20260809 03:28:56.979666 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:39467
--local_ip_for_outbound_sockets=127.9.46.194
--tserver_master_addrs=127.9.46.254:46637
--webserver_port=44219
--webserver_interface=127.9.46.194
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:28:56.989480 27277 log.cc:824] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:28:56.998594 27277 tablet_bootstrap.cc:492] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 03:28:56.999058 27277 tablet_bootstrap.cc:492] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Bootstrap complete.
I20260809 03:28:57.000694 27277 ts_tablet_manager.cc:1397] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Time spent bootstrapping tablet: real 0.034s user 0.026s sys 0.008s
I20260809 03:28:57.003952 27277 raft_consensus.cc:348] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:57.004529 27277 raft_consensus.cc:729] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b657f6beadf4ddba6409b756092968f, State: Initialized, Role: FOLLOWER
I20260809 03:28:57.005086 27277 consensus_queue.cc:260] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:57.006183 27277 ts_tablet_manager.cc:1428] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
W20260809 03:28:57.063445 27400 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:28:56.981563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:46435)
0809 03:28:56.982178 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:56.982242 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:56.983275 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:57.062283 (+ 79008us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":170,"threads_started":1}
W20260809 03:28:57.064107 27397 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:46637 (0 consecutive failures): Network error: Failed to ping master at 127.9.46.254:46637: Client connection negotiation failed: client connection to 127.9.46.254:46637: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 03:28:57.064977 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:57.064664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:46465)
0809 03:28:57.064821 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:57.064904 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 03:28:57.065749 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:57.065507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:49213)
0809 03:28:57.065631 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:57.065689 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 03:28:57.066001 27397 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:28:57.098171 27399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:57.098443 27399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:57.098523 27399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:57.102525 27399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:57.102591 27399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:57.102655 27399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:28:57.107937 27399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:39467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=44219
--tserver_master_addrs=127.9.46.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:57.109017 27399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:57.110081 27399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:57.119465 27408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:57.119522 27409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:57.119571 27399 server_base.cc:1034] running on GCE node
W20260809 03:28:57.119503 27411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:57.120240 27399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:57.120932 27399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:57.122144 27399 hybrid_clock.cc:648] HybridClock initialized: now 1786246137122118 us; error 45 us; skew 500 ppm
Aug 09 03:28:57 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246137, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:57.125159 27399 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:28:57.125959 27399 webserver.cc:460] Webserver started at http://127.9.46.194:44219/ using document root <none> and password file <none>
I20260809 03:28:57.126470 27399 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:57.126528 27399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:57.130223 27399 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:28:57.132133 27418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:57.133366 27399 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:28:57.133461 27399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "7f530e8fdc9d4798bd5d5457642946dc"
format_stamp: "Formatted at 2026-08-09 03:28:14 on dist-test-slave-65mx"
server_key: "caa211c7dfad659c1809550acc1e1e65"
server_key_iv: "0baba4b889b14d55121bedc6c0460b3a"
server_key_version: "encryptionkey@0"
I20260809 03:28:57.133961 27399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:57.152505 27399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:57.155897 27399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:57.156057 27399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:57.156661 27399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:28:57.157475 27399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:28:57.157522 27399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:57.157563 27399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:28:57.157586 27399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:57.166441 27399 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:39467
I20260809 03:28:57.166474 27542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:39467 every 8 connection(s)
I20260809 03:28:57.167512 27399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:28:57.169279 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 27399
I20260809 03:28:57.169716 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:41861
--local_ip_for_outbound_sockets=127.9.46.195
--tserver_master_addrs=127.9.46.254:46637
--webserver_port=35605
--webserver_interface=127.9.46.195
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:28:57.171823 27545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:57.169766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:44379)
0809 03:28:57.170730 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:57.170859 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":583,"threads_started":1}
W20260809 03:28:57.172312 27543 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:46637 (0 consecutive failures): Network error: Failed to ping master at 127.9.46.254:46637: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
I20260809 03:28:57.173017 27545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:57.172723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:56087)
0809 03:28:57.172868 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:57.172935 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 03:28:57.173804 27545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:57.173537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:53387)
0809 03:28:57.173665 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:57.173725 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 03:28:57.173987 27543 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 03:28:57.280051 27546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:28:57.280294 27546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:28:57.280336 27546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:28:57.284286 27546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:28:57.284344 27546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:28:57.284435 27546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:28:57.289570 27546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:41861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=35605
--tserver_master_addrs=127.9.46.254:46637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:28:57.290580 27546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:28:57.291587 27546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:28:57.301468 27551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:57.301581 27546 server_base.cc:1034] running on GCE node
W20260809 03:28:57.301486 27554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:28:57.301765 27552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:28:57.302196 27546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:28:57.302858 27546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:28:57.304029 27546 hybrid_clock.cc:648] HybridClock initialized: now 1786246137304014 us; error 28 us; skew 500 ppm
Aug 09 03:28:57 dist-test-slave-65mx krb5kdc[25959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246137, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:28:57.307029 27546 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:28:57.308019 27546 webserver.cc:460] Webserver started at http://127.9.46.195:35605/ using document root <none> and password file <none>
I20260809 03:28:57.308699 27546 fs_manager.cc:359] Metadata directory not provided
I20260809 03:28:57.308771 27546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:28:57.312770 27546 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:28:57.314821 27561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:57.316056 27546 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:28:57.316179 27546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "713e900cfbfe47f0ab21680d78a1d418"
format_stamp: "Formatted at 2026-08-09 03:28:15 on dist-test-slave-65mx"
server_key: "75425b83b82a486aefb2dfd2e8e4ac35"
server_key_iv: "34005c4bf8ecef2814b6ed9bd44093f9"
server_key_version: "encryptionkey@0"
I20260809 03:28:57.316776 27546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:28:57.329377 27546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:28:57.331938 27546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:28:57.332074 27546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:28:57.332566 27546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:28:57.333345 27546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:28:57.333392 27546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:57.333431 27546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:28:57.333447 27546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:28:57.342540 27546 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:41861
I20260809 03:28:57.342567 27684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:41861 every 8 connection(s)
I20260809 03:28:57.343475 27546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:28:57.347255 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:57.345600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:39099)
0809 03:28:57.346040 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:57.346535 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":107,"threads_started":1}
I20260809 03:28:57.347801 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 27546
W20260809 03:28:57.347857 27685 heartbeater.cc:644] Failed to heartbeat to 127.9.46.254:46637 (0 consecutive failures): Network error: Failed to ping master at 127.9.46.254:46637: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
I20260809 03:28:57.348673 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:57.348379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:51227)
0809 03:28:57.348513 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:57.348600 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 03:28:57.349422 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:57.349162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:45735)
0809 03:28:57.349324 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:57.349374 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 03:28:57.349582 27685 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 03:28:58.067137 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:58.066628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:54551)
0809 03:28:58.066894 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:58.066954 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260809 03:28:58.174986 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:58.174517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:41983)
0809 03:28:58.174770 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:58.174837 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":83,"threads_started":1}
I20260809 03:28:58.199901 27691 raft_consensus.cc:482] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:28:58.200156 27691 raft_consensus.cc:504] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:28:58.201350 27691 leader_election.cc:288] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861), 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467)
Aug 09 03:28:58 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246136, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.194@KRBTEST.COM
Aug 09 03:28:58 dist-test-slave-65mx krb5kdc[25959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246136, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.195@KRBTEST.COM
I20260809 03:28:58.215179 27692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:58.201739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:35527 (local address 127.9.46.195:41861)
0809 03:28:58.202132 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:58.202145 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:58.202183 (+ 38us) server_negotiation.cc:407] Connection header received
0809 03:28:58.202827 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:58.202834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:58.203175 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:58.203411 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:58.204745 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:58.205701 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:58.206873 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:58.207319 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:58.207492 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:58.210553 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:58.210557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:58.210567 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:58.210595 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:58.212763 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:58.213302 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:58.213309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:58.213314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:58.213447 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:58.213731 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:58.213734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:58.213735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:58.214163 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:58.214290 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:58.214717 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 03:28:58.214971 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":104,"threads_started":1}
I20260809 03:28:58.215164 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:58.201879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:39467 (local address 127.9.46.193:51893)
0809 03:28:58.202189 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:58.202205 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:58.202310 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:58.203727 (+ 1417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:58.203730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:58.204035 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:28:58.204495 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:58.204515 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:58.205848 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:58.205852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:58.206746 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:58.206755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:58.207446 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:58.207450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:58.207671 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:58.208380 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:28:58.208392 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:28:58.210018 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:28:58.212373 (+ 2355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:58.212379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:58.212393 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:58.212794 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:58.213192 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:58.213194 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:58.213196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:58.213408 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:58.214127 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:28:58.214134 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:28:58.214419 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 03:28:58.214602 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 03:28:58.214971 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":112,"spinlock_wait_cycles":2304,"thread_start_us":85,"threads_started":1}
I20260809 03:28:58.215210 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:58.202072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:51893 (local address 127.9.46.194:39467)
0809 03:28:58.202471 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0809 03:28:58.202489 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 03:28:58.202518 (+ 29us) server_negotiation.cc:407] Connection header received
0809 03:28:58.203121 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:28:58.203126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:28:58.203451 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:28:58.203738 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:28:58.204733 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:58.205695 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:58.206880 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:28:58.207320 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:28:58.207507 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:58.210259 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:28:58.210265 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:28:58.210276 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:28:58.210302 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:28:58.212245 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:58.212915 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:58.212921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:58.212927 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:58.213067 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:28:58.213543 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:28:58.213546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:28:58.213548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:28:58.214004 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:28:58.214105 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:28:58.214751 (+ 646us) server_negotiation.cc:299] Negotiation successful
0809 03:28:58.215021 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":90,"threads_started":1}
I20260809 03:28:58.215544 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:58.201584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:41861 (local address 127.9.46.193:35527)
0809 03:28:58.201719 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:58.201736 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:28:58.201843 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:28:58.203531 (+ 1688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:28:58.203537 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:28:58.204051 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:28:58.204496 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:58.204515 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:58.205846 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:58.205850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:58.206743 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:28:58.206752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:28:58.207445 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:28:58.207449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:28:58.207672 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:28:58.208338 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:28:58.208354 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:28:58.210355 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:28:58.212893 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:58.212896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:58.212898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:58.213173 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:58.213543 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:28:58.213546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:28:58.213548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:28:58.213640 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:28:58.214270 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:28:58.214273 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:28:58.214417 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 03:28:58.214601 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 03:28:58.215059 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":322,"spinlock_wait_cycles":32256}
W20260809 03:28:58.217789 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
W20260809 03:28:58.217996 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
I20260809 03:28:58.218066 27274 leader_election.cc:302] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b657f6beadf4ddba6409b756092968f; no voters: 713e900cfbfe47f0ab21680d78a1d418, 7f530e8fdc9d4798bd5d5457642946dc
I20260809 03:28:58.218351 27691 raft_consensus.cc:2726] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:28:58.350626 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:58.350171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:54481)
0809 03:28:58.350456 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:58.350515 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":90,"threads_started":1}
I20260809 03:28:59.068331 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:59.067918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:53951)
0809 03:28:59.068173 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:59.068233 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
I20260809 03:28:59.176092 27697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:59.175705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:48375)
0809 03:28:59.175948 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:59.176002 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
I20260809 03:28:59.351926 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:28:59.351474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:52915)
0809 03:28:59.351748 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 03:28:59.351809 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":76,"threads_started":1}
I20260809 03:29:00.069515 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:00.069104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:39109)
0809 03:29:00.069352 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:00.069410 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260809 03:29:00.177239 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:00.176842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:55923)
0809 03:29:00.177097 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:00.177152 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":75,"threads_started":1}
I20260809 03:29:00.267685 27701 raft_consensus.cc:482] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:00.267815 27701 raft_consensus.cc:504] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:29:00.268225 27701 leader_election.cc:288] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861), 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467)
W20260809 03:29:00.269148 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
W20260809 03:29:00.269261 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
I20260809 03:29:00.269300 27274 leader_election.cc:302] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b657f6beadf4ddba6409b756092968f; no voters: 713e900cfbfe47f0ab21680d78a1d418, 7f530e8fdc9d4798bd5d5457642946dc
I20260809 03:29:00.269438 27701 raft_consensus.cc:2726] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:29:00.353356 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:00.352895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:33425)
0809 03:29:00.353204 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:00.353264 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":107,"threads_started":1}
I20260809 03:29:01.070647 27703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:01.070210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:54281)
0809 03:29:01.070480 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:01.070534 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
I20260809 03:29:01.178426 27704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:01.177993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:43503)
0809 03:29:01.178287 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:01.178342 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":118,"threads_started":1}
I20260809 03:29:01.354579 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:01.354149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:47233)
0809 03:29:01.354423 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:01.354479 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":75,"threads_started":1}
I20260809 03:29:02.072041 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:02.071550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:40197)
0809 03:29:02.071880 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:02.071942 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":77,"threads_started":1}
I20260809 03:29:02.179675 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:02.179204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:37265)
0809 03:29:02.179493 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:02.179554 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":93,"threads_started":1}
I20260809 03:29:02.355968 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:02.355486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:43483)
0809 03:29:02.355792 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:02.355862 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":99,"threads_started":1}
I20260809 03:29:03.073213 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:03.072813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:49733)
0809 03:29:03.073060 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:03.073115 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":82,"threads_started":1}
I20260809 03:29:03.180989 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:03.180503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:37377)
0809 03:29:03.180828 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:03.180887 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":144,"threads_started":1}
I20260809 03:29:03.264290 27711 raft_consensus.cc:482] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:03.264423 27711 raft_consensus.cc:504] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:29:03.264889 27711 leader_election.cc:288] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861), 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467)
W20260809 03:29:03.265818 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
W20260809 03:29:03.265983 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
I20260809 03:29:03.266049 27274 leader_election.cc:302] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b657f6beadf4ddba6409b756092968f; no voters: 713e900cfbfe47f0ab21680d78a1d418, 7f530e8fdc9d4798bd5d5457642946dc
I20260809 03:29:03.266258 27711 raft_consensus.cc:2726] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:29:03.357304 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:03.356839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:52435)
0809 03:29:03.357142 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:03.357204 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":110,"threads_started":1}
I20260809 03:29:04.074424 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:04.073977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:51205)
0809 03:29:04.074258 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:04.074321 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":88,"threads_started":1}
I20260809 03:29:04.182334 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:04.181885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:46765)
0809 03:29:04.182187 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:04.182243 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":93,"threads_started":1}
I20260809 03:29:04.358640 27715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:04.358180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:42523)
0809 03:29:04.358484 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:04.358542 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":74,"threads_started":1}
I20260809 03:29:05.075659 27716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:05.075211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:53367)
0809 03:29:05.075508 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:05.075562 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":95,"threads_started":1}
I20260809 03:29:05.183578 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:05.183160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:54187)
0809 03:29:05.183424 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:05.183483 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":95,"threads_started":1}
I20260809 03:29:05.359800 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:05.359372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:58261)
0809 03:29:05.359648 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:05.359703 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260809 03:29:06.076964 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:06.076500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:42201)
0809 03:29:06.076792 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:06.076854 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":104,"threads_started":1}
I20260809 03:29:06.184803 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:06.184342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:46037)
0809 03:29:06.184653 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:06.184712 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
I20260809 03:29:06.313333 27722 raft_consensus.cc:482] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:06.313460 27722 raft_consensus.cc:504] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:29:06.313899 27722 leader_election.cc:288] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861), 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467)
W20260809 03:29:06.314805 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
W20260809 03:29:06.314986 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
I20260809 03:29:06.315055 27274 leader_election.cc:302] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b657f6beadf4ddba6409b756092968f; no voters: 713e900cfbfe47f0ab21680d78a1d418, 7f530e8fdc9d4798bd5d5457642946dc
I20260809 03:29:06.315239 27722 raft_consensus.cc:2726] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:29:06.361080 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:06.360632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:59991)
0809 03:29:06.360922 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:06.360984 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":101,"threads_started":1}
I20260809 03:29:07.078336 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:07.077917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:42545)
0809 03:29:07.078176 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:07.078239 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":81,"threads_started":1}
I20260809 03:29:07.186122 27725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:07.185631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:45259)
0809 03:29:07.185919 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:07.186003 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":115,"threads_started":1}
I20260809 03:29:07.362457 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:07.361976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:58995)
0809 03:29:07.362297 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:07.362360 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
I20260809 03:29:08.079563 27727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:08.079150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:32829)
0809 03:29:08.079408 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:08.079466 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":88,"threads_started":1}
I20260809 03:29:08.187328 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:08.186862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:34413)
0809 03:29:08.187148 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:08.187209 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":93,"threads_started":1}
I20260809 03:29:08.363658 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:08.363228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:49715)
0809 03:29:08.363499 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:08.363555 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260809 03:29:09.080865 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:09.080360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:58645)
0809 03:29:09.080670 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:09.080768 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":86,"threads_started":1}
I20260809 03:29:09.188752 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:09.188254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:56645)
0809 03:29:09.188523 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:09.188637 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":80,"threads_started":1}
I20260809 03:29:09.365095 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:09.364659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:39985)
0809 03:29:09.364923 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:09.364994 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":81,"threads_started":1}
I20260809 03:29:10.066411 27733 raft_consensus.cc:482] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:10.066547 27733 raft_consensus.cc:504] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:29:10.067075 27733 leader_election.cc:288] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861), 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467)
W20260809 03:29:10.067838 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
W20260809 03:29:10.068044 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
I20260809 03:29:10.068127 27274 leader_election.cc:302] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b657f6beadf4ddba6409b756092968f; no voters: 713e900cfbfe47f0ab21680d78a1d418, 7f530e8fdc9d4798bd5d5457642946dc
I20260809 03:29:10.068259 27733 raft_consensus.cc:2726] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:29:10.082010 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:10.081583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:41011)
0809 03:29:10.081857 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:10.081915 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":112,"threads_started":1}
I20260809 03:29:10.189940 27735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:10.189507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:56449)
0809 03:29:10.189787 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:10.189847 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
I20260809 03:29:10.366330 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:10.365903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:56549)
0809 03:29:10.366175 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:10.366232 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
I20260809 03:29:11.083221 27737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:11.082778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:49327)
0809 03:29:11.083048 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:11.083125 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
I20260809 03:29:11.191229 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:11.190760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:47497)
0809 03:29:11.191071 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:11.191134 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":107,"threads_started":1}
I20260809 03:29:11.367715 27741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:11.367214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.195:42417)
0809 03:29:11.367548 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:11.367614 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":151,"threads_started":1}
I20260809 03:29:11.899353 27742 raft_consensus.cc:482] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:11.899487 27742 raft_consensus.cc:504] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713e900cfbfe47f0ab21680d78a1d418" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41861 } } peers { permanent_uuid: "7b657f6beadf4ddba6409b756092968f" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44377 } } peers { permanent_uuid: "7f530e8fdc9d4798bd5d5457642946dc" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 39467 } }
I20260809 03:29:11.899957 27742 leader_election.cc:288] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861), 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467)
W20260809 03:29:11.900905 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f530e8fdc9d4798bd5d5457642946dc (127.9.46.194:39467): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
W20260809 03:29:11.901099 27274 leader_election.cc:341] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 713e900cfbfe47f0ab21680d78a1d418 (127.9.46.195:41861): Not found: Tablet not found: 7c73f86cde8d4332b6fe19fbc04a5e77
I20260809 03:29:11.901175 27274 leader_election.cc:302] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b657f6beadf4ddba6409b756092968f; no voters: 713e900cfbfe47f0ab21680d78a1d418, 7f530e8fdc9d4798bd5d5457642946dc
I20260809 03:29:11.901381 27742 raft_consensus.cc:2726] T 7c73f86cde8d4332b6fe19fbc04a5e77 P 7b657f6beadf4ddba6409b756092968f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 03:29:12.084755 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:12.084207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.193:39779)
0809 03:29:12.084582 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:12.084655 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":78,"threads_started":1}
I20260809 03:29:12.192617 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:12.192135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:46637 (local address 127.9.46.194:39961)
0809 03:29:12.192412 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:12.192466 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.254:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
I20260809 03:29:12.353060 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 27210
I20260809 03:29:12.361054 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 27399
I20260809 03:29:12.368252 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 27546
2026-08-09T03:29:12Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59322 ms)
[----------] 30 tests from SecurityITest (248232 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:12 dist-test-slave-65mx krb5kdc[27747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:12 dist-test-slave-65mx krb5kdc[27747](info): set up 2 sockets
Aug 09 03:29:12 dist-test-slave-65mx krb5kdc[27747](info): commencing operation
krb5kdc: starting...
W20260809 03:29:13.466250 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:13 dist-test-slave-65mx krb5kdc[27747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:13.599850 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39887
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:39887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:13.712387 27761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:13.712666 27761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:13.712706 27761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:13.716694 27761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:13.716758 27761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:13.716777 27761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:13.716794 27761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:13.716809 27761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:13.722112 27761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:39887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:13.723186 27761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:13.724220 27761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:13.733058 27768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:13.733049 27770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:13.733132 27761 server_base.cc:1034] running on GCE node
W20260809 03:29:13.733050 27767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:13.733963 27761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:13.734982 27761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:13.736141 27761 hybrid_clock.cc:648] HybridClock initialized: now 1786246153736124 us; error 30 us; skew 500 ppm
Aug 09 03:29:13 dist-test-slave-65mx krb5kdc[27747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246153, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:13.738864 27761 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:13.739707 27761 webserver.cc:460] Webserver started at http://127.9.46.254:34009/ using document root <none> and password file <none>
I20260809 03:29:13.740263 27761 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:13.740324 27761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:13.740620 27761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:13.742641 27761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "7c627a8be3f94b8c9f15554f43f200ee"
format_stamp: "Formatted at 2026-08-09 03:29:13 on dist-test-slave-65mx"
server_key: "88ff3748035b0f941ab53040521a74cb"
server_key_iv: "fd633ce85b3b62962b43ab5c19f36914"
server_key_version: "encryptionkey@0"
I20260809 03:29:13.743216 27761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "7c627a8be3f94b8c9f15554f43f200ee"
format_stamp: "Formatted at 2026-08-09 03:29:13 on dist-test-slave-65mx"
server_key: "88ff3748035b0f941ab53040521a74cb"
server_key_iv: "fd633ce85b3b62962b43ab5c19f36914"
server_key_version: "encryptionkey@0"
I20260809 03:29:13.746980 27761 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 03:29:13.749455 27777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:13.750646 27761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260809 03:29:13.750788 27761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "7c627a8be3f94b8c9f15554f43f200ee"
format_stamp: "Formatted at 2026-08-09 03:29:13 on dist-test-slave-65mx"
server_key: "88ff3748035b0f941ab53040521a74cb"
server_key_iv: "fd633ce85b3b62962b43ab5c19f36914"
server_key_version: "encryptionkey@0"
I20260809 03:29:13.750905 27761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:13.764029 27761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:13.771459 27761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:13.771672 27761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:13.779587 27761 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:39887
I20260809 03:29:13.779659 27834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:39887 every 8 connection(s)
I20260809 03:29:13.780767 27761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:13.781584 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 27761
I20260809 03:29:13.781733 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:13.782060 9403 external_mini_cluster.cc:1428] Setting key a2d51d62297125be309f1a6a78305ee1
I20260809 03:29:13.784080 27835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:13.789734 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee: Bootstrap starting.
Aug 09 03:29:13 dist-test-slave-65mx krb5kdc[27747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:13.792493 27835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:13.793339 27835 log.cc:824] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:13.795612 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee: No bootstrap required, opened a new log
I20260809 03:29:13.796245 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:13.783498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53152 (local address 127.9.46.254:39887)
0809 03:29:13.783952 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:13.783967 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:13.784000 (+ 33us) server_negotiation.cc:407] Connection header received
0809 03:29:13.784715 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:13.784733 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:13.785039 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:13.785361 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:13.786153 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:13.787137 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:13.787978 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:13.788456 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:13.788711 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:13.791437 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:13.791442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:13.791453 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:13.791480 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:13.793251 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:13.793774 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:13.793782 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:13.793790 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:13.793938 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:13.794249 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:13.794252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:13.794253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:13.794680 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:13.794827 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:13.795147 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 03:29:13.795449 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":168,"threads_started":1}
I20260809 03:29:13.799201 27835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c627a8be3f94b8c9f15554f43f200ee" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39887 } }
I20260809 03:29:13.799415 27835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:13.799495 27835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c627a8be3f94b8c9f15554f43f200ee, State: Initialized, Role: FOLLOWER
I20260809 03:29:13.799903 27835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c627a8be3f94b8c9f15554f43f200ee" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39887 } }
I20260809 03:29:13.800048 27835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:13.800112 27835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:13.800247 27835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:13.801230 27835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c627a8be3f94b8c9f15554f43f200ee" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39887 } }
I20260809 03:29:13.801566 27835 leader_election.cc:302] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c627a8be3f94b8c9f15554f43f200ee; no voters:
I20260809 03:29:13.801927 27835 leader_election.cc:288] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:13.802016 27841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:13.802263 27841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [term 1 LEADER]: Becoming Leader. State: Replica: 7c627a8be3f94b8c9f15554f43f200ee, State: Running, Role: LEADER
I20260809 03:29:13.802551 27841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c627a8be3f94b8c9f15554f43f200ee" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39887 } }
I20260809 03:29:13.803287 27835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:13.804256 27843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c627a8be3f94b8c9f15554f43f200ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c627a8be3f94b8c9f15554f43f200ee" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39887 } } }
I20260809 03:29:13.804297 27842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c627a8be3f94b8c9f15554f43f200ee. Latest consensus state: current_term: 1 leader_uuid: "7c627a8be3f94b8c9f15554f43f200ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c627a8be3f94b8c9f15554f43f200ee" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39887 } } }
I20260809 03:29:13.804512 27842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:13.804502 27843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:13.805425 27851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:13.808233 27851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:13.813576 27851 catalog_manager.cc:1347] Generated new cluster ID: 03a6dc4e27764cf29961b6f2ccd0f62e
I20260809 03:29:13.813668 27851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:13.828218 27851 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:13.829186 27851 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:13.838548 27851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee: Generated new TSK 0
I20260809 03:29:13.839285 27851 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:29:13.883337 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39887
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:14.013931 27864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:14.014186 27864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:14.014228 27864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:14.018251 27864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:14.018301 27864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:14.018356 27864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:29:14.023674 27864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:14.024670 27864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:14.025601 27864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:14.034744 27870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:14.034744 27869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:14.034775 27872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:14.034986 27864 server_base.cc:1034] running on GCE node
I20260809 03:29:14.035548 27864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:14.036176 27864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:14.037362 27864 hybrid_clock.cc:648] HybridClock initialized: now 1786246154037343 us; error 36 us; skew 500 ppm
Aug 09 03:29:14 dist-test-slave-65mx krb5kdc[27747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:14.040076 27864 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:29:14.040970 27864 webserver.cc:460] Webserver started at http://127.9.46.193:35805/ using document root <none> and password file <none>
I20260809 03:29:14.041473 27864 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:14.041529 27864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:14.041687 27864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:14.043597 27864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "bf52ad6db4b94ce9952f71b5854208a8"
format_stamp: "Formatted at 2026-08-09 03:29:14 on dist-test-slave-65mx"
server_key: "5a8b29daaa0e1370131708d533eca975"
server_key_iv: "5dab176fec1a32b6fbdcb143f04e70a0"
server_key_version: "encryptionkey@0"
I20260809 03:29:14.044143 27864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "bf52ad6db4b94ce9952f71b5854208a8"
format_stamp: "Formatted at 2026-08-09 03:29:14 on dist-test-slave-65mx"
server_key: "5a8b29daaa0e1370131708d533eca975"
server_key_iv: "5dab176fec1a32b6fbdcb143f04e70a0"
server_key_version: "encryptionkey@0"
I20260809 03:29:14.047755 27864 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 03:29:14.050199 27880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:14.051378 27864 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:29:14.051501 27864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "bf52ad6db4b94ce9952f71b5854208a8"
format_stamp: "Formatted at 2026-08-09 03:29:14 on dist-test-slave-65mx"
server_key: "5a8b29daaa0e1370131708d533eca975"
server_key_iv: "5dab176fec1a32b6fbdcb143f04e70a0"
server_key_version: "encryptionkey@0"
I20260809 03:29:14.051610 27864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:14.070611 27864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:14.073832 27864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:14.074072 27864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:14.074770 27864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:14.075661 27864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:14.075713 27864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:14.075793 27864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:14.075836 27864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:14.086117 27864 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:42303
I20260809 03:29:14.086138 28012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:42303 every 8 connection(s)
I20260809 03:29:14.087078 27864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:29:14.090879 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 27864
I20260809 03:29:14.091020 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:29:14.091413 9403 external_mini_cluster.cc:1428] Setting key 70a103f08024395a393d22ff19c6835f
Aug 09 03:29:14 dist-test-slave-65mx krb5kdc[27747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:14.100692 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.089041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:41249 (local address 127.9.46.254:39887)
0809 03:29:14.089180 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:14.089184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:14.090078 (+ 894us) server_negotiation.cc:407] Connection header received
0809 03:29:14.091229 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:14.091233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:14.091283 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:14.091358 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:14.092618 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.093151 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.094281 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.094518 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.094631 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.097105 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:14.097108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:14.097110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:14.097134 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:14.098639 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:14.099148 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:14.099152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:14.099155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:14.099238 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:14.099576 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:14.099581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:14.099582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:14.099763 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:14.099879 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:14.100355 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 03:29:14.100484 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:29:14.101455 28015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.089423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:39887 (local address 127.9.46.193:41249)
0809 03:29:14.089938 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:14.089979 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:14.090944 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:14.091555 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:14.091564 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:14.092000 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:14.092449 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.092462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.093289 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.093300 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.094153 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.094162 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.094679 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.094684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.094848 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.095446 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:14.095461 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:14.096923 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:14.098765 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:14.098770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:14.098784 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:14.099041 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:14.099333 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:14.099337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:14.099340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:14.099479 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:14.099877 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:14.099884 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:14.100126 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 03:29:14.100313 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 03:29:14.100655 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":201,"threads_started":1}
I20260809 03:29:14.102674 28013 heartbeater.cc:347] Connected to a master server at 127.9.46.254:39887
I20260809 03:29:14.102885 28013 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
I20260809 03:29:14.103472 28013 heartbeater.cc:511] Master 127.9.46.254:39887 requested a full tablet report, sending...
I20260809 03:29:14.105197 27795 ts_manager.cc:194] Registered new tserver with Master: bf52ad6db4b94ce9952f71b5854208a8 (127.9.46.193:42303)
I20260809 03:29:14.106308 27795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:41249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:29:14.119856 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39887
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:14.232749 28018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:14.232995 28018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:14.233039 28018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:14.237044 28018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:14.237093 28018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:14.237150 28018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:29:14.242147 28018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:14.243058 28018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:14.243949 28018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:14.253336 28026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:14.253302 28023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:14.253301 28024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:14.253394 28018 server_base.cc:1034] running on GCE node
I20260809 03:29:14.253911 28018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:14.254613 28018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:14.255805 28018 hybrid_clock.cc:648] HybridClock initialized: now 1786246154255798 us; error 30 us; skew 500 ppm
Aug 09 03:29:14 dist-test-slave-65mx krb5kdc[27747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:14.258563 28018 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:29:14.259333 28018 webserver.cc:460] Webserver started at http://127.9.46.194:39101/ using document root <none> and password file <none>
I20260809 03:29:14.259828 28018 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:14.259886 28018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:14.260042 28018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:14.262060 28018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "b2b739a055584311bdf49c1c14fb8fec"
format_stamp: "Formatted at 2026-08-09 03:29:14 on dist-test-slave-65mx"
server_key: "ee5811b653930891483c3c2f2705f280"
server_key_iv: "ae80931b681ba39076876e72abb8068a"
server_key_version: "encryptionkey@0"
I20260809 03:29:14.262563 28018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "b2b739a055584311bdf49c1c14fb8fec"
format_stamp: "Formatted at 2026-08-09 03:29:14 on dist-test-slave-65mx"
server_key: "ee5811b653930891483c3c2f2705f280"
server_key_iv: "ae80931b681ba39076876e72abb8068a"
server_key_version: "encryptionkey@0"
I20260809 03:29:14.266237 28018 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:29:14.268484 28033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:14.269719 28018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:14.269824 28018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "b2b739a055584311bdf49c1c14fb8fec"
format_stamp: "Formatted at 2026-08-09 03:29:14 on dist-test-slave-65mx"
server_key: "ee5811b653930891483c3c2f2705f280"
server_key_iv: "ae80931b681ba39076876e72abb8068a"
server_key_version: "encryptionkey@0"
I20260809 03:29:14.269912 28018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:14.285352 28018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:14.288511 28018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:14.288699 28018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:14.289219 28018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:14.289963 28018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:14.290046 28018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:14.290108 28018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:14.290146 28018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:14.299691 28018 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:34595
I20260809 03:29:14.299710 28158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:34595 every 8 connection(s)
I20260809 03:29:14.300536 28018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:29:14.307490 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 28018
I20260809 03:29:14.307633 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:29:14.307988 9403 external_mini_cluster.cc:1428] Setting key c4723b9c79b922bb621616050d2fd8aa
Aug 09 03:29:14 dist-test-slave-65mx krb5kdc[27747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:14.314421 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.302281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:34885 (local address 127.9.46.254:39887)
0809 03:29:14.302411 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:14.302415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:14.303180 (+ 765us) server_negotiation.cc:407] Connection header received
0809 03:29:14.304167 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:14.304170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:14.304229 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:14.304353 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:14.305686 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.306232 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.307374 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.307614 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.307743 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.310429 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:14.310432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:14.310434 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:14.310456 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:14.312149 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:14.312707 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:14.312712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:14.312715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:14.312820 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:14.313166 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:14.313171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:14.313176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:14.313427 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:14.313532 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:14.314011 (+ 479us) server_negotiation.cc:299] Negotiation successful
0809 03:29:14.314168 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:29:14.315032 28161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.302593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:39887 (local address 127.9.46.194:34885)
0809 03:29:14.303029 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:14.303072 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:14.303949 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:14.304518 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:14.304524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:14.305076 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:14.305534 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.305546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.306359 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.306363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.307240 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.307249 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.307812 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.307818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.308033 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.308661 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:14.308678 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:14.310266 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:14.312274 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:14.312278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:14.312288 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:14.312537 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:14.312925 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:14.312928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:14.312929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:14.313063 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:14.313528 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:14.313533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:14.313763 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 03:29:14.313907 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 03:29:14.314156 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260809 03:29:14.316238 28159 heartbeater.cc:347] Connected to a master server at 127.9.46.254:39887
I20260809 03:29:14.316481 28159 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:14.317003 28159 heartbeater.cc:511] Master 127.9.46.254:39887 requested a full tablet report, sending...
I20260809 03:29:14.318058 27795 ts_manager.cc:194] Registered new tserver with Master: b2b739a055584311bdf49c1c14fb8fec (127.9.46.194:34595)
I20260809 03:29:14.318635 27795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:34885
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:29:14.339059 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39887
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:14.450197 28164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:14.450443 28164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:14.450484 28164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:14.455075 28164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:14.455147 28164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:14.455206 28164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:29:14.460517 28164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:14.461550 28164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:14.462497 28164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:14.471984 28170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:14.471947 28169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:14.471958 28172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:14.472352 28164 server_base.cc:1034] running on GCE node
I20260809 03:29:14.472805 28164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:14.473474 28164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:14.474665 28164 hybrid_clock.cc:648] HybridClock initialized: now 1786246154474653 us; error 24 us; skew 500 ppm
Aug 09 03:29:14 dist-test-slave-65mx krb5kdc[27747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:14.477458 28164 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:29:14.478677 28164 webserver.cc:460] Webserver started at http://127.9.46.195:46857/ using document root <none> and password file <none>
I20260809 03:29:14.479270 28164 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:14.479367 28164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:14.479575 28164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:14.481662 28164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "bf4a8ddd7a9643fd8768d97bd78bb030"
format_stamp: "Formatted at 2026-08-09 03:29:14 on dist-test-slave-65mx"
server_key: "10de2970b0ea99100ee42127f2b0c332"
server_key_iv: "bb1eb08c05c4ccf05bf0a1ee1384d12d"
server_key_version: "encryptionkey@0"
I20260809 03:29:14.482232 28164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "bf4a8ddd7a9643fd8768d97bd78bb030"
format_stamp: "Formatted at 2026-08-09 03:29:14 on dist-test-slave-65mx"
server_key: "10de2970b0ea99100ee42127f2b0c332"
server_key_iv: "bb1eb08c05c4ccf05bf0a1ee1384d12d"
server_key_version: "encryptionkey@0"
I20260809 03:29:14.485906 28164 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:29:14.488250 28179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:14.489463 28164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:14.489580 28164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "bf4a8ddd7a9643fd8768d97bd78bb030"
format_stamp: "Formatted at 2026-08-09 03:29:14 on dist-test-slave-65mx"
server_key: "10de2970b0ea99100ee42127f2b0c332"
server_key_iv: "bb1eb08c05c4ccf05bf0a1ee1384d12d"
server_key_version: "encryptionkey@0"
I20260809 03:29:14.489688 28164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:14.506815 28164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:14.510138 28164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:14.510350 28164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:14.510951 28164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:14.511770 28164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:14.511817 28164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:14.511885 28164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:14.511926 28164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:14.520447 28164 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:38379
I20260809 03:29:14.520545 28309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:38379 every 8 connection(s)
I20260809 03:29:14.521368 28164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:29:14.526052 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 28164
I20260809 03:29:14.526175 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:29:14.526492 9403 external_mini_cluster.cc:1428] Setting key 3af4035a9ac0b33a24ce0b0dd89ae918
Aug 09 03:29:14 dist-test-slave-65mx krb5kdc[27747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:14.534643 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.523151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:39383 (local address 127.9.46.254:39887)
0809 03:29:14.523303 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:14.523307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:14.524097 (+ 790us) server_negotiation.cc:407] Connection header received
0809 03:29:14.525077 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:14.525080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:14.525139 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:14.525252 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:14.526487 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.527232 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.528698 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.528916 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.528999 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.531436 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:14.531439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:14.531441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:14.531465 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:14.532942 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:14.533382 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:14.533385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:14.533386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:14.533458 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:14.533765 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:14.533767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:14.533769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:14.533933 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:14.534007 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:14.534432 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 03:29:14.534526 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 03:29:14.535455 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.523499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:39887 (local address 127.9.46.195:39383)
0809 03:29:14.523952 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:14.523993 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:14.524869 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:14.525416 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:14.525422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:14.525769 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:14.526339 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.526355 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.527370 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.527374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.528572 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.528584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.529045 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.529047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.529184 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.529787 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:14.529802 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:14.531287 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:14.533038 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:14.533042 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:14.533053 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:14.533297 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:14.533536 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:14.533538 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:14.533539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:14.533686 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:14.534019 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:14.534024 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:14.534238 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 03:29:14.534387 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 03:29:14.534626 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":125,"threads_started":1}
I20260809 03:29:14.536617 28310 heartbeater.cc:347] Connected to a master server at 127.9.46.254:39887
I20260809 03:29:14.536854 28310 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:14.537348 28310 heartbeater.cc:511] Master 127.9.46.254:39887 requested a full tablet report, sending...
I20260809 03:29:14.538357 27795 ts_manager.cc:194] Registered new tserver with Master: bf4a8ddd7a9643fd8768d97bd78bb030 (127.9.46.195:38379)
I20260809 03:29:14.538960 27795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:39383
I20260809 03:29:14.541505 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:29:14.550369 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.543437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53166 (local address 127.9.46.254:39887)
0809 03:29:14.543552 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:14.543555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:14.543759 (+ 204us) server_negotiation.cc:407] Connection header received
0809 03:29:14.543845 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:14.543847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:14.543881 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:14.543946 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:14.544636 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.545114 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.545920 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.546121 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.546213 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.547491 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:14.547494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:14.547495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:14.547512 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:14.548860 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:14.549231 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:14.549233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:14.549234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:14.549306 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:14.549537 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:14.549539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:14.549540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:14.549733 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:14.549835 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:14.550066 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 03:29:14.550153 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 03:29:14.553246 27794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:29:14.556600 27794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:29:14.570233 28321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.564632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:34595 (local address 127.0.0.1:36498)
0809 03:29:14.564972 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:14.565043 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:14.565178 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:14.565692 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:14.565700 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:14.565743 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:14.565902 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.565914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.567190 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.567193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.568516 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.568526 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.569668 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.569682 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.569821 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.569861 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 03:29:14.569962 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 03:29:14.570077 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":16,"spinlock_wait_cycles":896,"thread_start_us":81,"threads_started":1}
I20260809 03:29:14.570233 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.564797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.9.46.194:34595)
0809 03:29:14.565182 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:14.565194 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:14.565221 (+ 27us) server_negotiation.cc:407] Connection header received
0809 03:29:14.565313 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:14.565318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:14.565518 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:14.565691 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:14.566041 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.567076 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.568706 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.569537 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.569786 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.569956 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:14.570006 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 03:29:14.570090 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":162,"threads_started":1}
I20260809 03:29:14.570721 28323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.565371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:38379 (local address 127.0.0.1:42016)
0809 03:29:14.565635 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:14.565649 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:14.565725 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:14.566806 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:14.566811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:14.566833 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:14.566952 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.566962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.568372 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.568375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.569529 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.569538 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.570309 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.570312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.570429 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.570453 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:29:14.570514 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 03:29:14.570576 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260809 03:29:14.570990 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.565549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.9.46.195:38379)
0809 03:29:14.566208 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:14.566219 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:14.566242 (+ 23us) server_negotiation.cc:407] Connection header received
0809 03:29:14.566312 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:14.566319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:14.566498 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:14.566641 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:14.567086 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.568227 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.569652 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.570184 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.570450 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.570592 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:14.570639 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 03:29:14.570721 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":180,"threads_started":1}
I20260809 03:29:14.572671 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.566047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:42303 (local address 127.0.0.1:57838)
0809 03:29:14.566917 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:14.566932 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:14.567032 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:14.567467 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:14.567470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:14.567486 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:14.567602 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.567610 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.568625 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.568630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.570727 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.570739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.572224 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.572228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.572327 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.572348 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:29:14.572429 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 03:29:14.572497 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":29,"thread_start_us":102,"threads_started":1}
I20260809 03:29:14.573079 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.566226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57838 (local address 127.9.46.193:42303)
0809 03:29:14.566785 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:14.566800 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:14.567030 (+ 230us) server_negotiation.cc:407] Connection header received
0809 03:29:14.567145 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:14.567152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:14.567328 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:14.567472 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:14.567701 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.568481 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.570986 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.572091 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.572222 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.572359 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:14.572826 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 03:29:14.572924 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":67,"threads_started":1}
I20260809 03:29:14.573398 28232 tablet_service.cc:1467] Processing CreateTablet for tablet 33b5432e752c4bbd80d46464f6247e1f (DEFAULT_TABLE table=test-table [id=377e6a757fb1462ba17e78ebf38703bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:14.573401 28086 tablet_service.cc:1467] Processing CreateTablet for tablet 33b5432e752c4bbd80d46464f6247e1f (DEFAULT_TABLE table=test-table [id=377e6a757fb1462ba17e78ebf38703bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:14.574501 28086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33b5432e752c4bbd80d46464f6247e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:14.574503 28232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33b5432e752c4bbd80d46464f6247e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:14.575433 27934 tablet_service.cc:1467] Processing CreateTablet for tablet 33b5432e752c4bbd80d46464f6247e1f (DEFAULT_TABLE table=test-table [id=377e6a757fb1462ba17e78ebf38703bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:14.576246 27934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33b5432e752c4bbd80d46464f6247e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:14.582422 28330 tablet_bootstrap.cc:492] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8: Bootstrap starting.
I20260809 03:29:14.582504 28327 tablet_bootstrap.cc:492] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: Bootstrap starting.
I20260809 03:29:14.582808 28329 tablet_bootstrap.cc:492] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: Bootstrap starting.
I20260809 03:29:14.584667 28330 tablet_bootstrap.cc:654] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:14.584667 28329 tablet_bootstrap.cc:654] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:14.584717 28327 tablet_bootstrap.cc:654] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:14.585420 28327 log.cc:824] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:14.585911 28329 log.cc:824] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:14.585911 28330 log.cc:824] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:14.587448 28327 tablet_bootstrap.cc:492] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: No bootstrap required, opened a new log
I20260809 03:29:14.587641 28327 ts_tablet_manager.cc:1397] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 03:29:14.587730 28330 tablet_bootstrap.cc:492] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8: No bootstrap required, opened a new log
I20260809 03:29:14.587867 28329 tablet_bootstrap.cc:492] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: No bootstrap required, opened a new log
I20260809 03:29:14.587903 28330 ts_tablet_manager.cc:1397] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 03:29:14.588065 28329 ts_tablet_manager.cc:1397] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 03:29:14.591024 28329 raft_consensus.cc:348] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b739a055584311bdf49c1c14fb8fec" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 34595 } } peers { permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 } } peers { permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 } }
I20260809 03:29:14.591275 28329 raft_consensus.cc:374] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:14.591332 28329 raft_consensus.cc:729] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf4a8ddd7a9643fd8768d97bd78bb030, State: Initialized, Role: FOLLOWER
I20260809 03:29:14.591452 28330 raft_consensus.cc:348] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b739a055584311bdf49c1c14fb8fec" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 34595 } } peers { permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 } } peers { permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 } }
I20260809 03:29:14.591452 28327 raft_consensus.cc:348] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b739a055584311bdf49c1c14fb8fec" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 34595 } } peers { permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 } } peers { permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 } }
I20260809 03:29:14.591694 28330 raft_consensus.cc:374] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:14.591694 28327 raft_consensus.cc:374] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:14.591753 28330 raft_consensus.cc:729] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf52ad6db4b94ce9952f71b5854208a8, State: Initialized, Role: FOLLOWER
I20260809 03:29:14.591763 28327 raft_consensus.cc:729] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b739a055584311bdf49c1c14fb8fec, State: Initialized, Role: FOLLOWER
I20260809 03:29:14.591806 28329 consensus_queue.cc:260] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b739a055584311bdf49c1c14fb8fec" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 34595 } } peers { permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 } } peers { permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 } }
I20260809 03:29:14.592195 28330 consensus_queue.cc:260] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b739a055584311bdf49c1c14fb8fec" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 34595 } } peers { permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 } } peers { permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 } }
I20260809 03:29:14.592191 28327 consensus_queue.cc:260] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b739a055584311bdf49c1c14fb8fec" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 34595 } } peers { permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 } } peers { permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 } }
I20260809 03:29:14.592530 28310 heartbeater.cc:503] Master 127.9.46.254:39887 was elected leader, sending a full tablet report...
I20260809 03:29:14.592862 28329 ts_tablet_manager.cc:1428] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:29:14.593132 28327 ts_tablet_manager.cc:1428] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:29:14.593132 28330 ts_tablet_manager.cc:1428] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 03:29:14.593462 28013 heartbeater.cc:503] Master 127.9.46.254:39887 was elected leader, sending a full tablet report...
I20260809 03:29:14.594439 28159 heartbeater.cc:503] Master 127.9.46.254:39887 was elected leader, sending a full tablet report...
I20260809 03:29:14.769135 28337 raft_consensus.cc:482] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:14.769356 28337 raft_consensus.cc:504] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b739a055584311bdf49c1c14fb8fec" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 34595 } } peers { permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 } } peers { permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 } }
I20260809 03:29:14.770535 28337 leader_election.cc:288] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf4a8ddd7a9643fd8768d97bd78bb030 (127.9.46.195:38379), bf52ad6db4b94ce9952f71b5854208a8 (127.9.46.193:42303)
W20260809 03:29:14.772352 28311 tablet.cc:2369] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:29:14.774618 28161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.770778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:38379 (local address 127.9.46.194:54945)
0809 03:29:14.770968 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:14.770984 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:14.771081 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:14.771360 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:14.771363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:14.771380 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:14.771502 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.771508 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.772263 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.772266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.773368 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.773377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.774068 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.774075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.774220 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.774293 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 03:29:14.774375 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:29:14.774472 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":62}
I20260809 03:29:14.774610 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.770879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:54945 (local address 127.9.46.195:38379)
0809 03:29:14.771057 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:14.771061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:14.771077 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:29:14.771218 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:14.771221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:14.771263 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:14.771349 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:14.771612 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.772102 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.773486 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.773933 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.774029 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.774106 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:14.774404 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 03:29:14.774491 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 03:29:14.774756 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.770905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:50807 (local address 127.9.46.193:42303)
0809 03:29:14.771145 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:14.771151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:14.771167 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:29:14.771324 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:14.771327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:14.771379 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:14.771465 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:14.771735 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.772256 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.773557 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.774032 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.774144 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.774193 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:14.774548 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 03:29:14.774620 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260809 03:29:14.775086 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.770798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:42303 (local address 127.9.46.194:50807)
0809 03:29:14.771061 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:14.771076 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:14.771197 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:14.771474 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:14.771477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:14.771493 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:14.771619 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.771624 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.772375 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.772380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.773451 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:14.773463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.774227 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.774230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:14.774412 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.774432 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 03:29:14.774889 (+ 457us) client_negotiation.cc:240] Negotiation successful
0809 03:29:14.774962 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":64,"thread_start_us":104,"threads_started":1}
I20260809 03:29:14.775400 28253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b5432e752c4bbd80d46464f6247e1f" candidate_uuid: "b2b739a055584311bdf49c1c14fb8fec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" is_pre_election: true
I20260809 03:29:14.775746 28253 raft_consensus.cc:2454] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b739a055584311bdf49c1c14fb8fec in term 0.
I20260809 03:29:14.775720 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b5432e752c4bbd80d46464f6247e1f" candidate_uuid: "b2b739a055584311bdf49c1c14fb8fec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf52ad6db4b94ce9952f71b5854208a8" is_pre_election: true
I20260809 03:29:14.775978 27956 raft_consensus.cc:2454] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b739a055584311bdf49c1c14fb8fec in term 0.
I20260809 03:29:14.776252 28035 leader_election.cc:302] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2b739a055584311bdf49c1c14fb8fec, bf4a8ddd7a9643fd8768d97bd78bb030; no voters:
I20260809 03:29:14.776525 28337 raft_consensus.cc:2781] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:29:14.776669 28337 raft_consensus.cc:482] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:29:14.776724 28337 raft_consensus.cc:3037] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:14.777801 28337 raft_consensus.cc:504] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b739a055584311bdf49c1c14fb8fec" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 34595 } } peers { permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 } } peers { permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 } }
I20260809 03:29:14.778196 28337 leader_election.cc:288] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [CANDIDATE]: Term 1 election: Requested vote from peers bf4a8ddd7a9643fd8768d97bd78bb030 (127.9.46.195:38379), bf52ad6db4b94ce9952f71b5854208a8 (127.9.46.193:42303)
I20260809 03:29:14.778564 27956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b5432e752c4bbd80d46464f6247e1f" candidate_uuid: "b2b739a055584311bdf49c1c14fb8fec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf52ad6db4b94ce9952f71b5854208a8"
I20260809 03:29:14.778645 28253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b5432e752c4bbd80d46464f6247e1f" candidate_uuid: "b2b739a055584311bdf49c1c14fb8fec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030"
I20260809 03:29:14.778715 27956 raft_consensus.cc:3037] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:14.778785 28253 raft_consensus.cc:3037] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:14.780037 28253 raft_consensus.cc:2454] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b739a055584311bdf49c1c14fb8fec in term 1.
I20260809 03:29:14.780037 27956 raft_consensus.cc:2454] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b739a055584311bdf49c1c14fb8fec in term 1.
I20260809 03:29:14.780387 28035 leader_election.cc:302] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2b739a055584311bdf49c1c14fb8fec, bf4a8ddd7a9643fd8768d97bd78bb030; no voters:
I20260809 03:29:14.780603 28337 raft_consensus.cc:2781] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:14.780874 28337 raft_consensus.cc:686] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 1 LEADER]: Becoming Leader. State: Replica: b2b739a055584311bdf49c1c14fb8fec, State: Running, Role: LEADER
I20260809 03:29:14.781210 28337 consensus_queue.cc:237] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b739a055584311bdf49c1c14fb8fec" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 34595 } } peers { permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 } } peers { permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 } }
I20260809 03:29:14.785943 27795 catalog_manager.cc:5495] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec reported cstate change: term changed from 0 to 1, leader changed from <none> to b2b739a055584311bdf49c1c14fb8fec (127.9.46.194). New cstate: current_term: 1 leader_uuid: "b2b739a055584311bdf49c1c14fb8fec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b739a055584311bdf49c1c14fb8fec" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 34595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 } health_report { overall_health: UNKNOWN } } }
W20260809 03:29:14.801692 28160 tablet.cc:2369] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:29:14.806734 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.803224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36506 (local address 127.9.46.194:34595)
0809 03:29:14.803344 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:14.803348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:14.803361 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:29:14.803480 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:14.803482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:14.803535 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:14.803656 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:14.803918 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.804479 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.805441 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:14.805651 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:14.805733 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:14.806005 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:14.806426 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:14.806517 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:14.806573 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 03:29:14.806623 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:29:14.812486 28253 raft_consensus.cc:1261] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030 [term 1 FOLLOWER]: Refusing update from remote peer b2b739a055584311bdf49c1c14fb8fec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:14.812486 27956 raft_consensus.cc:1261] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8 [term 1 FOLLOWER]: Refusing update from remote peer b2b739a055584311bdf49c1c14fb8fec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:14.813274 28337 consensus_queue.cc:1035] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf52ad6db4b94ce9952f71b5854208a8" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:14.813534 28339 consensus_queue.cc:1035] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf4a8ddd7a9643fd8768d97bd78bb030" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:14.821095 28344 mvcc.cc:205] Tried to move back new op lower bound from 7316464250104561664 to 7316464249986514944. Current Snapshot: MvccSnapshot[applied={T|T < 7316464250104561664}]
I20260809 03:29:14.821990 28343 mvcc.cc:205] Tried to move back new op lower bound from 7316464250104561664 to 7316464249986514944. Current Snapshot: MvccSnapshot[applied={T|T < 7316464250104561664}]
I20260809 03:29:14.831149 27793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:14.831305 27793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:14.833820 27793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c627a8be3f94b8c9f15554f43f200ee: Sending DeleteTablet for 3 replicas of tablet 33b5432e752c4bbd80d46464f6247e1f
I20260809 03:29:14.834719 27934 tablet_service.cc:1514] Processing DeleteTablet for tablet 33b5432e752c4bbd80d46464f6247e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:14 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:57838
I20260809 03:29:14.834697 28086 tablet_service.cc:1514] Processing DeleteTablet for tablet 33b5432e752c4bbd80d46464f6247e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:14 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:36498
I20260809 03:29:14.835230 28351 tablet_replica.cc:330] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: stopping tablet replica
I20260809 03:29:14.835230 28352 tablet_replica.cc:330] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8: stopping tablet replica
I20260809 03:29:14.835275 28232 tablet_service.cc:1514] Processing DeleteTablet for tablet 33b5432e752c4bbd80d46464f6247e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:14 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:42016
I20260809 03:29:14.835548 28351 raft_consensus.cc:2229] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:29:14.835723 28352 raft_consensus.cc:2229] T 33b5432e752c4bbd80d46464f6247e1f P bf52ad6db4b94ce9952f71b5854208a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:29:14.835964 28351 raft_consensus.cc:2258] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:14.836381 28353 tablet_replica.cc:330] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: stopping tablet replica
I20260809 03:29:14.836469 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 27864
I20260809 03:29:14.836747 28353 raft_consensus.cc:2229] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:29:14.837041 28353 raft_consensus.cc:2258] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:14.838415 28351 ts_tablet_manager.cc:1905] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:29:14.838846 28353 ts_tablet_manager.cc:1905] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:29:14.841559 28351 ts_tablet_manager.cc:1918] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:29:14.841660 28351 log.cc:1196] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/33b5432e752c4bbd80d46464f6247e1f
I20260809 03:29:14.841946 28351 ts_tablet_manager.cc:1939] T 33b5432e752c4bbd80d46464f6247e1f P b2b739a055584311bdf49c1c14fb8fec: Deleting consensus metadata
I20260809 03:29:14.842689 28353 ts_tablet_manager.cc:1918] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:29:14.842806 28353 log.cc:1196] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/33b5432e752c4bbd80d46464f6247e1f
I20260809 03:29:14.843091 27780 catalog_manager.cc:4841] TS b2b739a055584311bdf49c1c14fb8fec (127.9.46.194:34595): tablet 33b5432e752c4bbd80d46464f6247e1f (table test-table [id=377e6a757fb1462ba17e78ebf38703bb]) successfully deleted
I20260809 03:29:14.843220 28353 ts_tablet_manager.cc:1939] T 33b5432e752c4bbd80d46464f6247e1f P bf4a8ddd7a9643fd8768d97bd78bb030: Deleting consensus metadata
I20260809 03:29:14.844583 27780 catalog_manager.cc:4841] TS bf4a8ddd7a9643fd8768d97bd78bb030 (127.9.46.195:38379): tablet 33b5432e752c4bbd80d46464f6247e1f (table test-table [id=377e6a757fb1462ba17e78ebf38703bb]) successfully deleted
W20260809 03:29:14.851266 27778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:42303 (error 108)
I20260809 03:29:14.851382 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 28018
I20260809 03:29:14.854296 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:14.853951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:42303 (local address 127.0.0.1:57850)
0809 03:29:14.854095 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:14.854211 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:42303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 03:29:14.854470 27778 catalog_manager.cc:4584] TS bf52ad6db4b94ce9952f71b5854208a8 (127.9.46.193:42303): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33b5432e752c4bbd80d46464f6247e1f: Network error: Client connection negotiation failed: client connection to 127.9.46.193:42303: connect: Connection refused (error 111)
I20260809 03:29:14.858441 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 28164
I20260809 03:29:14.865644 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 27761
2026-08-09T03:29:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2498 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T03:29:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:14Z Disabled control of system clock
I20260809 03:29:14.899536 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:44881
--webserver_interface=127.9.46.254
--webserver_port=0
--builtin_ntp_servers=127.9.46.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:44881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:29:15.009152 28358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:15.009409 28358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:15.009473 28358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:15.013423 28358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:15.013473 28358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:15.013489 28358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:15.013505 28358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:15.013553 28358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:15.018307 28358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:44881
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:44881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:15.019333 28358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:15.020687 28358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:15.029143 28364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:15.029202 28358 server_base.cc:1034] running on GCE node
W20260809 03:29:15.029119 28363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:15.029143 28366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:15.029978 28358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:15.031045 28358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:15.032240 28358 hybrid_clock.cc:648] HybridClock initialized: now 1786246155032223 us; error 30 us; skew 500 ppm
I20260809 03:29:15.034087 28358 webserver.cc:460] Webserver started at http://127.9.46.254:44753/ using document root <none> and password file <none>
I20260809 03:29:15.034667 28358 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:15.034763 28358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:15.034996 28358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:15.037055 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "34f7b512b0cb45a29d866b109454d865"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "3a9168f7140212cc3f9571e02e873aa8"
server_key_iv: "70b4f2480cb7a96bd0e692a6718c7e31"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.037618 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "34f7b512b0cb45a29d866b109454d865"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "3a9168f7140212cc3f9571e02e873aa8"
server_key_iv: "70b4f2480cb7a96bd0e692a6718c7e31"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.041083 28358 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:29:15.043285 28373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:15.044373 28358 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:15.044518 28358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "34f7b512b0cb45a29d866b109454d865"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "3a9168f7140212cc3f9571e02e873aa8"
server_key_iv: "70b4f2480cb7a96bd0e692a6718c7e31"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.044678 28358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:15.058045 28358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:15.058688 28358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:15.058881 28358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:15.065747 28358 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:44881
I20260809 03:29:15.065757 28430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:44881 every 8 connection(s)
I20260809 03:29:15.066668 28358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:15.069684 28431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:15.074892 28431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865: Bootstrap starting.
I20260809 03:29:15.075654 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 28358
I20260809 03:29:15.075788 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:15.076103 9403 external_mini_cluster.cc:1428] Setting key 10bb42dd3e2838e615bf5bca04ad1082
I20260809 03:29:15.077889 28431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:15.078622 28431 log.cc:824] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:15.080518 28431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865: No bootstrap required, opened a new log
I20260809 03:29:15.084079 28431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f7b512b0cb45a29d866b109454d865" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44881 } }
I20260809 03:29:15.084556 28431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:15.084669 28431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34f7b512b0cb45a29d866b109454d865, State: Initialized, Role: FOLLOWER
I20260809 03:29:15.085201 28431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f7b512b0cb45a29d866b109454d865" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44881 } }
I20260809 03:29:15.085335 28431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:15.085423 28431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:15.085537 28431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:15.085807 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.077356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.9.46.254:44881)
0809 03:29:15.078037 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.078053 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.078088 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:29:15.078761 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.078790 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.079282 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.079659 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:15.080087 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.081135 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.082041 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.082534 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.082708 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.083670 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:15.083676 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:15.083691 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:15.084070 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:15.084126 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:15.084145 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:15.084408 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.084622 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.084914 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":131,"threads_started":1}
I20260809 03:29:15.086583 28431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f7b512b0cb45a29d866b109454d865" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44881 } }
I20260809 03:29:15.086911 28431 leader_election.cc:302] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34f7b512b0cb45a29d866b109454d865; no voters:
I20260809 03:29:15.087253 28431 leader_election.cc:288] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:15.087323 28437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:15.087584 28437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [term 1 LEADER]: Becoming Leader. State: Replica: 34f7b512b0cb45a29d866b109454d865, State: Running, Role: LEADER
I20260809 03:29:15.087885 28437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f7b512b0cb45a29d866b109454d865" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44881 } }
I20260809 03:29:15.088608 28431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:15.089569 28438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34f7b512b0cb45a29d866b109454d865. Latest consensus state: current_term: 1 leader_uuid: "34f7b512b0cb45a29d866b109454d865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f7b512b0cb45a29d866b109454d865" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44881 } } }
I20260809 03:29:15.089617 28439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34f7b512b0cb45a29d866b109454d865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34f7b512b0cb45a29d866b109454d865" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44881 } } }
I20260809 03:29:15.089723 28438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:15.089740 28439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:15.091126 28448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:15.093770 28448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:15.099458 28448 catalog_manager.cc:1347] Generated new cluster ID: 2a5958c3d52a4e6495d5dfdb724c6b81
I20260809 03:29:15.099566 28448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:15.106655 28448 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:15.107892 28448 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:15.118713 28448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865: Generated new TSK 0
I20260809 03:29:15.119498 28448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:15.127060 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44881
--builtin_ntp_servers=127.9.46.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:29:15.238391 28458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:15.238669 28458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:15.238737 28458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:15.242735 28458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:15.242821 28458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:15.242905 28458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:29:15.248409 28458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:15.249531 28458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:15.250873 28458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:15.260229 28464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:15.260226 28466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:15.260219 28463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:15.260599 28458 server_base.cc:1034] running on GCE node
I20260809 03:29:15.261068 28458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:15.261756 28458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:15.263005 28458 hybrid_clock.cc:648] HybridClock initialized: now 1786246155262951 us; error 37 us; skew 500 ppm
I20260809 03:29:15.265105 28458 webserver.cc:460] Webserver started at http://127.9.46.193:35131/ using document root <none> and password file <none>
I20260809 03:29:15.265715 28458 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:15.265816 28458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:15.266078 28458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:15.268321 28458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "142454b427844d39aed0e04a82158d61"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "094630ea805d3b54fc7aa88dd94ed669"
server_key_iv: "ab6b0ac39da49148c223e10e4970c1cc"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.268999 28458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "142454b427844d39aed0e04a82158d61"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "094630ea805d3b54fc7aa88dd94ed669"
server_key_iv: "ab6b0ac39da49148c223e10e4970c1cc"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.273219 28458 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:29:15.275812 28472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:15.277189 28458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:15.277338 28458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "142454b427844d39aed0e04a82158d61"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "094630ea805d3b54fc7aa88dd94ed669"
server_key_iv: "ab6b0ac39da49148c223e10e4970c1cc"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.277462 28458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:15.294775 28458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:15.295531 28458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:15.295753 28458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:15.296403 28458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:15.297514 28458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:15.297598 28458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:15.297678 28458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:15.297729 28458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:15.307300 28458 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:43775
I20260809 03:29:15.307327 28598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:43775 every 8 connection(s)
I20260809 03:29:15.308269 28458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:29:15.314414 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 28458
I20260809 03:29:15.314579 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:29:15.314987 9403 external_mini_cluster.cc:1428] Setting key 236c1ac0aa77117ed65082a7f364fc43
I20260809 03:29:15.317209 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44881
--builtin_ntp_servers=127.9.46.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:29:15.318409 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.310150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:34591 (local address 127.9.46.254:44881)
0809 03:29:15.310296 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.310306 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.311472 (+ 1166us) server_negotiation.cc:407] Connection header received
0809 03:29:15.313040 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.313042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.313088 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.313166 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:15.314084 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.314693 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.316219 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.316560 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.316644 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.317962 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:15.317965 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:15.317967 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:15.317979 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:15.317997 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:15.318005 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:15.318063 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.318252 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.318304 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:29:15.319530 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.310561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44881 (local address 127.9.46.193:34591)
0809 03:29:15.311310 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:15.311355 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:15.312833 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:15.313361 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:15.313367 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:15.313447 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:15.313949 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.313961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.314842 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.314846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.316070 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.316080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.316698 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.316703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.316899 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.317647 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:29:15.317651 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:15.317707 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:29:15.317799 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:15.318091 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:15.318104 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:15.318113 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 03:29:15.318246 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 03:29:15.318545 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":150,"threads_started":1}
I20260809 03:29:15.320780 28600 heartbeater.cc:347] Connected to a master server at 127.9.46.254:44881
I20260809 03:29:15.321017 28600 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:15.321614 28600 heartbeater.cc:511] Master 127.9.46.254:44881 requested a full tablet report, sending...
I20260809 03:29:15.323112 28392 ts_manager.cc:194] Registered new tserver with Master: 142454b427844d39aed0e04a82158d61 (127.9.46.193:43775)
I20260809 03:29:15.324241 28392 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.193:34591
W20260809 03:29:15.438973 28603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:15.439222 28603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:15.439289 28603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:15.443338 28603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:15.443392 28603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:15.443493 28603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:29:15.448261 28603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:15.449347 28603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:15.450700 28603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:15.460014 28609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:15.460007 28612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:15.460041 28603 server_base.cc:1034] running on GCE node
W20260809 03:29:15.460007 28610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:15.460737 28603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:15.461290 28603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:15.462448 28603 hybrid_clock.cc:648] HybridClock initialized: now 1786246155462437 us; error 24 us; skew 500 ppm
I20260809 03:29:15.464154 28603 webserver.cc:460] Webserver started at http://127.9.46.194:37311/ using document root <none> and password file <none>
I20260809 03:29:15.464763 28603 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:15.464828 28603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:15.465067 28603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:15.467111 28603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "1f480a4d75b24ee7acd2ac25a884830c"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "1a4a2b8fbf3c5817f6eae78cdca603ac"
server_key_iv: "45936e524e6b484c9e54a934d6b93c1f"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.467657 28603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "1f480a4d75b24ee7acd2ac25a884830c"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "1a4a2b8fbf3c5817f6eae78cdca603ac"
server_key_iv: "45936e524e6b484c9e54a934d6b93c1f"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.471194 28603 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 03:29:15.473538 28618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:15.474684 28603 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:15.474817 28603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "1f480a4d75b24ee7acd2ac25a884830c"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "1a4a2b8fbf3c5817f6eae78cdca603ac"
server_key_iv: "45936e524e6b484c9e54a934d6b93c1f"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.474928 28603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:15.494581 28603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:15.495301 28603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:15.495505 28603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:15.496102 28603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:15.497124 28603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:15.497197 28603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:15.497269 28603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:15.497323 28603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:15.506188 28603 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:40355
I20260809 03:29:15.506213 28739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:40355 every 8 connection(s)
I20260809 03:29:15.507057 28603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:29:15.516160 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.508931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:34985 (local address 127.9.46.254:44881)
0809 03:29:15.509074 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.509077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.510053 (+ 976us) server_negotiation.cc:407] Connection header received
0809 03:29:15.511466 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.511470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.511531 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.511647 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:15.512513 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.513067 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.514207 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.514427 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.514539 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.515675 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:15.515677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:15.515679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:15.515691 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:15.515709 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:15.515717 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:15.515796 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.515954 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.516017 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 03:29:15.517148 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.509258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44881 (local address 127.9.46.194:34985)
0809 03:29:15.509890 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:15.509937 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:15.511255 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:15.511813 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:15.511820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:15.511901 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:15.512364 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.512376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.513212 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.513221 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.514075 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.514084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.514584 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.514588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.514759 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.515409 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:29:15.515411 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:15.515449 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:29:15.515508 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:15.515798 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:15.515812 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:15.515820 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:29:15.515951 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 03:29:15.516230 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":111,"threads_started":1}
I20260809 03:29:15.517283 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 28603
I20260809 03:29:15.517426 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:29:15.517725 9403 external_mini_cluster.cc:1428] Setting key 306001a59516723ddcc0cda6f68c2986
I20260809 03:29:15.518350 28740 heartbeater.cc:347] Connected to a master server at 127.9.46.254:44881
I20260809 03:29:15.518591 28740 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:15.519081 28740 heartbeater.cc:511] Master 127.9.46.254:44881 requested a full tablet report, sending...
I20260809 03:29:15.519654 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44881
--builtin_ntp_servers=127.9.46.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:29:15.520089 28392 ts_manager.cc:194] Registered new tserver with Master: 1f480a4d75b24ee7acd2ac25a884830c (127.9.46.194:40355)
I20260809 03:29:15.520711 28392 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.194:34985
W20260809 03:29:15.631095 28744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:15.631363 28744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:15.631402 28744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:15.635401 28744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:15.635457 28744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:15.635512 28744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:29:15.640326 28744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:15.641512 28744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:15.642953 28744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:15.652662 28750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:15.652700 28752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:15.652873 28744 server_base.cc:1034] running on GCE node
W20260809 03:29:15.652729 28749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:15.653390 28744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:15.654088 28744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:15.655278 28744 hybrid_clock.cc:648] HybridClock initialized: now 1786246155655264 us; error 31 us; skew 500 ppm
I20260809 03:29:15.657256 28744 webserver.cc:460] Webserver started at http://127.9.46.195:37459/ using document root <none> and password file <none>
I20260809 03:29:15.657841 28744 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:15.657936 28744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:15.658169 28744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:15.660177 28744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "093a22cb110448518d2e25c7c909b452"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "87d676e838129bf53d8c12c348653d40"
server_key_iv: "3eaadd21d5615c00c6d0404a5b3001d6"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.660796 28744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "093a22cb110448518d2e25c7c909b452"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "87d676e838129bf53d8c12c348653d40"
server_key_iv: "3eaadd21d5615c00c6d0404a5b3001d6"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.664345 28744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 03:29:15.666952 28759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:15.668080 28744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:15.668226 28744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "093a22cb110448518d2e25c7c909b452"
format_stamp: "Formatted at 2026-08-09 03:29:15 on dist-test-slave-65mx"
server_key: "87d676e838129bf53d8c12c348653d40"
server_key_iv: "3eaadd21d5615c00c6d0404a5b3001d6"
server_key_version: "encryptionkey@0"
I20260809 03:29:15.668341 28744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:15.687724 28744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:15.688529 28744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:15.688786 28744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:15.689436 28744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:15.690467 28744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:15.690541 28744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:15.690608 28744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:15.690659 28744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:15.699798 28744 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:41285
I20260809 03:29:15.699826 28880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:41285 every 8 connection(s)
I20260809 03:29:15.700791 28744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:29:15.706888 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 28744
I20260809 03:29:15.706993 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:29:15.707283 9403 external_mini_cluster.cc:1428] Setting key adfc5cc21238b1df17a638e9624f176a
I20260809 03:29:15.710778 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.702702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:52397 (local address 127.9.46.254:44881)
0809 03:29:15.702877 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.702881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.703939 (+ 1058us) server_negotiation.cc:407] Connection header received
0809 03:29:15.705517 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.705521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.705582 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.705691 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:15.706659 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.707282 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.708778 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.709073 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.709216 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.710367 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:15.710370 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:15.710372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:15.710384 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:15.710404 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:15.710412 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:15.710470 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.710613 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.710667 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 03:29:15.711615 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.703099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44881 (local address 127.9.46.195:52397)
0809 03:29:15.703774 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:15.703819 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:15.705287 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:15.705876 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:15.705883 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:15.705976 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:15.706499 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.706512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.707484 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.707488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.708650 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.708661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.709271 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.709274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.709418 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.710106 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:29:15.710107 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:15.710149 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:29:15.710225 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:15.710486 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:15.710493 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:15.710497 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:29:15.710591 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 03:29:15.710813 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":104,"threads_started":1}
I20260809 03:29:15.712669 28881 heartbeater.cc:347] Connected to a master server at 127.9.46.254:44881
I20260809 03:29:15.712903 28881 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:15.713407 28881 heartbeater.cc:511] Master 127.9.46.254:44881 requested a full tablet report, sending...
I20260809 03:29:15.714395 28392 ts_manager.cc:194] Registered new tserver with Master: 093a22cb110448518d2e25c7c909b452 (127.9.46.195:41285)
I20260809 03:29:15.715000 28392 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.195:52397
I20260809 03:29:15.721647 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:29:15.727630 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.723650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53236 (local address 127.9.46.254:44881)
0809 03:29:15.723831 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.723836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.723982 (+ 146us) server_negotiation.cc:407] Connection header received
0809 03:29:15.724185 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.724189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.724242 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.724328 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:15.724644 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.725171 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.726021 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.726243 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.726351 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.727193 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:15.727196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:15.727199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:15.727212 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:15.727239 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:15.727247 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:15.727328 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.727421 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.727479 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 03:29:15.730424 28392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:29:15.733469 28392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:29:15.748426 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.743067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.9.46.193:43775)
0809 03:29:15.743508 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.743521 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.743546 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:29:15.743614 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.743619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.743820 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.743970 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:15.744200 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.745351 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.746919 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.747657 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.747776 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.747987 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.748118 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.748244 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":162,"threads_started":1}
I20260809 03:29:15.748421 28894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.742024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:41285 (local address 127.0.0.1:35748)
0809 03:29:15.742687 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:15.742702 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:15.742837 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:15.744128 (+ 1291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:15.744132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:15.744150 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:15.744229 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.744234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.745272 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.745277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.746832 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.746842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.748007 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.748013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.748120 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.748146 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 03:29:15.748210 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 03:29:15.748276 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":93,"spinlock_wait_cycles":3968,"thread_start_us":130,"threads_started":1}
I20260809 03:29:15.748420 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.742651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:43775 (local address 127.0.0.1:46190)
0809 03:29:15.743167 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:15.743182 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:15.743270 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:15.743957 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:15.743960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:15.743978 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:15.744059 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.744065 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.745452 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.745455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.746818 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.746822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.747776 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.747780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.747901 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.747939 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 03:29:15.748077 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 03:29:15.748215 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":68,"thread_start_us":105,"threads_started":1}
I20260809 03:29:15.748641 28896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.742000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33452 (local address 127.9.46.194:40355)
0809 03:29:15.742539 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.742554 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.742717 (+ 163us) server_negotiation.cc:407] Connection header received
0809 03:29:15.743032 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.743079 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.743296 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.743432 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:15.743893 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.745182 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.746822 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.747649 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.747798 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.747990 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.748328 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.748431 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":148,"threads_started":1}
I20260809 03:29:15.749229 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.742474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.9.46.195:41285)
0809 03:29:15.743620 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.743632 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.743658 (+ 26us) server_negotiation.cc:407] Connection header received
0809 03:29:15.743716 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.743719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.743884 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.744025 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:15.744354 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.745148 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.746972 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.747729 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.747821 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.748933 (+ 1112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.748982 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.749069 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":192,"threads_started":1}
I20260809 03:29:15.749603 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.741783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:40355 (local address 127.0.0.1:33452)
0809 03:29:15.742564 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:15.742615 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:15.742839 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:15.743582 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:15.743587 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:15.743610 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:15.743755 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.743761 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.745323 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.745326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.746668 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.746678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.748069 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.748074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.748168 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.748187 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 03:29:15.748835 (+ 648us) client_negotiation.cc:240] Negotiation successful
0809 03:29:15.748907 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":263,"spinlock_wait_cycles":4736,"thread_start_us":91,"threads_started":1}
I20260809 03:29:15.751992 28667 tablet_service.cc:1467] Processing CreateTablet for tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e (DEFAULT_TABLE table=test-table [id=595d3a3d7a8f469a9bf76868753d4605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:15.752084 28810 tablet_service.cc:1467] Processing CreateTablet for tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e (DEFAULT_TABLE table=test-table [id=595d3a3d7a8f469a9bf76868753d4605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:15.752110 28525 tablet_service.cc:1467] Processing CreateTablet for tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e (DEFAULT_TABLE table=test-table [id=595d3a3d7a8f469a9bf76868753d4605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:15.753140 28810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:15.753140 28525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:15.753141 28667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:15.759511 28900 tablet_bootstrap.cc:492] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c: Bootstrap starting.
I20260809 03:29:15.760893 28902 tablet_bootstrap.cc:492] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61: Bootstrap starting.
I20260809 03:29:15.761955 28900 tablet_bootstrap.cc:654] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:15.762665 28902 tablet_bootstrap.cc:654] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:15.762665 28900 log.cc:824] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:15.763309 28902 log.cc:824] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:15.764004 28901 tablet_bootstrap.cc:492] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: Bootstrap starting.
I20260809 03:29:15.764830 28900 tablet_bootstrap.cc:492] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c: No bootstrap required, opened a new log
I20260809 03:29:15.765027 28900 ts_tablet_manager.cc:1397] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:29:15.765774 28902 tablet_bootstrap.cc:492] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61: No bootstrap required, opened a new log
I20260809 03:29:15.765957 28902 ts_tablet_manager.cc:1397] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 03:29:15.766304 28901 tablet_bootstrap.cc:654] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:15.767067 28901 log.cc:824] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:15.768535 28900 raft_consensus.cc:348] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f480a4d75b24ee7acd2ac25a884830c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 40355 } } peers { permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 } } peers { permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 } }
I20260809 03:29:15.768829 28900 raft_consensus.cc:374] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:15.768901 28900 raft_consensus.cc:729] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f480a4d75b24ee7acd2ac25a884830c, State: Initialized, Role: FOLLOWER
I20260809 03:29:15.769071 28901 tablet_bootstrap.cc:492] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: No bootstrap required, opened a new log
I20260809 03:29:15.769253 28901 ts_tablet_manager.cc:1397] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:29:15.769418 28900 consensus_queue.cc:260] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f480a4d75b24ee7acd2ac25a884830c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 40355 } } peers { permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 } } peers { permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 } }
I20260809 03:29:15.769778 28902 raft_consensus.cc:348] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f480a4d75b24ee7acd2ac25a884830c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 40355 } } peers { permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 } } peers { permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 } }
I20260809 03:29:15.770045 28740 heartbeater.cc:503] Master 127.9.46.254:44881 was elected leader, sending a full tablet report...
I20260809 03:29:15.770092 28902 raft_consensus.cc:374] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:15.770146 28902 raft_consensus.cc:729] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142454b427844d39aed0e04a82158d61, State: Initialized, Role: FOLLOWER
I20260809 03:29:15.770568 28902 consensus_queue.cc:260] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f480a4d75b24ee7acd2ac25a884830c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 40355 } } peers { permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 } } peers { permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 } }
I20260809 03:29:15.771322 28900 ts_tablet_manager.cc:1428] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:29:15.771476 28902 ts_tablet_manager.cc:1428] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 03:29:15.771857 28600 heartbeater.cc:503] Master 127.9.46.254:44881 was elected leader, sending a full tablet report...
I20260809 03:29:15.773371 28901 raft_consensus.cc:348] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f480a4d75b24ee7acd2ac25a884830c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 40355 } } peers { permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 } } peers { permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 } }
I20260809 03:29:15.773586 28901 raft_consensus.cc:374] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:15.773646 28901 raft_consensus.cc:729] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093a22cb110448518d2e25c7c909b452, State: Initialized, Role: FOLLOWER
I20260809 03:29:15.774086 28901 consensus_queue.cc:260] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f480a4d75b24ee7acd2ac25a884830c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 40355 } } peers { permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 } } peers { permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 } }
I20260809 03:29:15.774762 28881 heartbeater.cc:503] Master 127.9.46.254:44881 was elected leader, sending a full tablet report...
I20260809 03:29:15.775058 28901 ts_tablet_manager.cc:1428] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260809 03:29:15.809497 28601 tablet.cc:2369] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:29:15.890085 28907 raft_consensus.cc:482] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:15.890436 28907 raft_consensus.cc:504] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f480a4d75b24ee7acd2ac25a884830c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 40355 } } peers { permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 } } peers { permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 } }
I20260809 03:29:15.891836 28907 leader_election.cc:288] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 093a22cb110448518d2e25c7c909b452 (127.9.46.195:41285), 142454b427844d39aed0e04a82158d61 (127.9.46.193:43775)
I20260809 03:29:15.895792 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.892104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:40083 (local address 127.9.46.195:41285)
0809 03:29:15.892272 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.892277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.892294 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:29:15.892431 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.892435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.892514 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.892635 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:15.892967 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.893684 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.894803 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.895223 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.895352 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.895404 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.895589 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.895669 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 03:29:15.895789 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.892017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:41285 (local address 127.9.46.194:40083)
0809 03:29:15.892213 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:15.892229 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:15.892318 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:15.892652 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:15.892655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:15.892714 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:15.892847 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.892859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.893797 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.893801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.894670 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.894679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.895347 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.895351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.895449 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.895480 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 03:29:15.895540 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 03:29:15.895627 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":23}
I20260809 03:29:15.895812 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.892283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:47629 (local address 127.9.46.193:43775)
0809 03:29:15.892418 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.892423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.892606 (+ 183us) server_negotiation.cc:407] Connection header received
0809 03:29:15.892751 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.892754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.892807 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.892892 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:15.893209 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.893795 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.894857 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.895229 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.895359 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.895404 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.895634 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.895709 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:29:15.895807 28910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.892070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:43775 (local address 127.9.46.194:47629)
0809 03:29:15.892486 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:15.892500 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:15.892638 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:15.892933 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:15.892936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:15.892956 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:15.893080 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.893087 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.893905 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.893908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.894699 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:15.894708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.895371 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.895374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:15.895483 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.895500 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 03:29:15.895597 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 03:29:15.895660 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":26,"thread_start_us":142,"threads_started":1}
I20260809 03:29:15.896459 28833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9e4b5aacc4542a6b3ceb3dbbd47f59e" candidate_uuid: "1f480a4d75b24ee7acd2ac25a884830c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093a22cb110448518d2e25c7c909b452" is_pre_election: true
I20260809 03:29:15.896617 28547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9e4b5aacc4542a6b3ceb3dbbd47f59e" candidate_uuid: "1f480a4d75b24ee7acd2ac25a884830c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142454b427844d39aed0e04a82158d61" is_pre_election: true
I20260809 03:29:15.896807 28833 raft_consensus.cc:2454] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f480a4d75b24ee7acd2ac25a884830c in term 0.
I20260809 03:29:15.896937 28547 raft_consensus.cc:2454] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f480a4d75b24ee7acd2ac25a884830c in term 0.
I20260809 03:29:15.897359 28621 leader_election.cc:302] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093a22cb110448518d2e25c7c909b452, 1f480a4d75b24ee7acd2ac25a884830c; no voters:
I20260809 03:29:15.897632 28907 raft_consensus.cc:2781] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:29:15.897717 28907 raft_consensus.cc:482] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:29:15.897760 28907 raft_consensus.cc:3037] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:15.898768 28907 raft_consensus.cc:504] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f480a4d75b24ee7acd2ac25a884830c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 40355 } } peers { permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 } } peers { permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 } }
I20260809 03:29:15.899104 28907 leader_election.cc:288] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [CANDIDATE]: Term 1 election: Requested vote from peers 093a22cb110448518d2e25c7c909b452 (127.9.46.195:41285), 142454b427844d39aed0e04a82158d61 (127.9.46.193:43775)
I20260809 03:29:15.899391 28833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9e4b5aacc4542a6b3ceb3dbbd47f59e" candidate_uuid: "1f480a4d75b24ee7acd2ac25a884830c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093a22cb110448518d2e25c7c909b452"
I20260809 03:29:15.899480 28547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9e4b5aacc4542a6b3ceb3dbbd47f59e" candidate_uuid: "1f480a4d75b24ee7acd2ac25a884830c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "142454b427844d39aed0e04a82158d61"
I20260809 03:29:15.899529 28833 raft_consensus.cc:3037] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:15.899603 28547 raft_consensus.cc:3037] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:15.900621 28547 raft_consensus.cc:2454] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f480a4d75b24ee7acd2ac25a884830c in term 1.
I20260809 03:29:15.900636 28833 raft_consensus.cc:2454] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f480a4d75b24ee7acd2ac25a884830c in term 1.
I20260809 03:29:15.900942 28619 leader_election.cc:302] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142454b427844d39aed0e04a82158d61, 1f480a4d75b24ee7acd2ac25a884830c; no voters:
I20260809 03:29:15.901139 28907 raft_consensus.cc:2781] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:15.901404 28907 raft_consensus.cc:686] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 1 LEADER]: Becoming Leader. State: Replica: 1f480a4d75b24ee7acd2ac25a884830c, State: Running, Role: LEADER
I20260809 03:29:15.901724 28907 consensus_queue.cc:237] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f480a4d75b24ee7acd2ac25a884830c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 40355 } } peers { permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 } } peers { permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 } }
I20260809 03:29:15.904747 28392 catalog_manager.cc:5495] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f480a4d75b24ee7acd2ac25a884830c (127.9.46.194). New cstate: current_term: 1 leader_uuid: "1f480a4d75b24ee7acd2ac25a884830c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f480a4d75b24ee7acd2ac25a884830c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 40355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 } health_report { overall_health: UNKNOWN } } }
I20260809 03:29:15.936748 28896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.933201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33454 (local address 127.9.46.194:40355)
0809 03:29:15.933364 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:15.933369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:15.933387 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:29:15.933475 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:15.933477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:15.933532 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:15.933647 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:15.933915 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.934415 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.935263 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:15.935484 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:15.935597 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:15.935902 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:15.936307 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:15.936436 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:15.936495 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 03:29:15.936571 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:29:15.942493 28547 raft_consensus.cc:1261] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 142454b427844d39aed0e04a82158d61 [term 1 FOLLOWER]: Refusing update from remote peer 1f480a4d75b24ee7acd2ac25a884830c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:15.942493 28833 raft_consensus.cc:1261] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452 [term 1 FOLLOWER]: Refusing update from remote peer 1f480a4d75b24ee7acd2ac25a884830c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:15.943240 28907 consensus_queue.cc:1035] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [LEADER]: Connected to new peer: Peer: permanent_uuid: "142454b427844d39aed0e04a82158d61" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:15.943423 28911 consensus_queue.cc:1035] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [LEADER]: Connected to new peer: Peer: permanent_uuid: "093a22cb110448518d2e25c7c909b452" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 41285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:15.950968 28914 mvcc.cc:205] Tried to move back new op lower bound from 7316464254733205504 to 7316464254575927296. Current Snapshot: MvccSnapshot[applied={T|T < 7316464254733205504}]
W20260809 03:29:15.951709 28882 tablet.cc:2369] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:29:15.954219 28916 mvcc.cc:205] Tried to move back new op lower bound from 7316464254733205504 to 7316464254575927296. Current Snapshot: MvccSnapshot[applied={T|T < 7316464254733205504}]
I20260809 03:29:15.959776 28392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:15.959951 28392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:15.962486 28392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34f7b512b0cb45a29d866b109454d865: Sending DeleteTablet for 3 replicas of tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e
I20260809 03:29:15.963230 28667 tablet_service.cc:1514] Processing DeleteTablet for tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:15 UTC) from {username='slave'} at 127.0.0.1:33452
I20260809 03:29:15.963250 28525 tablet_service.cc:1514] Processing DeleteTablet for tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:15 UTC) from {username='slave'} at 127.0.0.1:46190
I20260809 03:29:15.963323 28810 tablet_service.cc:1514] Processing DeleteTablet for tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:15 UTC) from {username='slave'} at 127.0.0.1:35748
I20260809 03:29:15.963792 28926 tablet_replica.cc:330] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: stopping tablet replica
I20260809 03:29:15.964143 28926 raft_consensus.cc:2229] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:29:15.964558 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 28458
I20260809 03:29:15.964526 28926 raft_consensus.cc:2258] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:15.969591 28926 ts_tablet_manager.cc:1905] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:29:15.971441 28924 tablet_replica.cc:330] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c: stopping tablet replica
I20260809 03:29:15.971774 28924 raft_consensus.cc:2229] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:29:15.972033 28926 ts_tablet_manager.cc:1918] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:29:15.972141 28926 log.cc:1196] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/a9e4b5aacc4542a6b3ceb3dbbd47f59e
I20260809 03:29:15.972141 28924 raft_consensus.cc:2258] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 1f480a4d75b24ee7acd2ac25a884830c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:15.972479 28926 ts_tablet_manager.cc:1939] T a9e4b5aacc4542a6b3ceb3dbbd47f59e P 093a22cb110448518d2e25c7c909b452: Deleting consensus metadata
I20260809 03:29:15.973706 28376 catalog_manager.cc:4841] TS 093a22cb110448518d2e25c7c909b452 (127.9.46.195:41285): tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e (table test-table [id=595d3a3d7a8f469a9bf76868753d4605]) successfully deleted
W20260809 03:29:15.975754 28374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:43775 (error 108)
I20260809 03:29:15.976351 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 28603
I20260809 03:29:15.979766 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.979385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:43775 (local address 127.0.0.1:46198)
0809 03:29:15.979553 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:15.979650 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:43775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 03:29:15.979950 28374 catalog_manager.cc:4584] TS 142454b427844d39aed0e04a82158d61 (127.9.46.193:43775): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a9e4b5aacc4542a6b3ceb3dbbd47f59e: Network error: Client connection negotiation failed: client connection to 127.9.46.193:43775: connect: Connection refused (error 111)
I20260809 03:29:15.985179 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 28744
I20260809 03:29:15.987422 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:15.985363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:40355 (local address 127.0.0.1:33464)
0809 03:29:15.987248 (+ 1885us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:15.987314 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.194:40355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1776}
I20260809 03:29:15.991961 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 28358
2026-08-09T03:29:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1124 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:16 dist-test-slave-65mx krb5kdc[28930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:16 dist-test-slave-65mx krb5kdc[28930](info): set up 2 sockets
Aug 09 03:29:16 dist-test-slave-65mx krb5kdc[28930](info): commencing operation
krb5kdc: starting...
W20260809 03:29:17.073594 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:17 dist-test-slave-65mx krb5kdc[28930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:17.203882 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41285
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:41285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:17.313910 28944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:17.314186 28944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:17.314227 28944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:17.318202 28944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:17.318262 28944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:17.318281 28944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:17.318298 28944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:17.318313 28944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:17.323547 28944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:41285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:41285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:17.324570 28944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:17.325520 28944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:17.333791 28951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:17.333781 28950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:17.333854 28944 server_base.cc:1034] running on GCE node
W20260809 03:29:17.333781 28953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:17.334578 28944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:17.335625 28944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:17.336790 28944 hybrid_clock.cc:648] HybridClock initialized: now 1786246157336772 us; error 30 us; skew 500 ppm
Aug 09 03:29:17 dist-test-slave-65mx krb5kdc[28930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246157, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:17.339450 28944 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:17.340363 28944 webserver.cc:460] Webserver started at http://127.9.46.254:33557/ using document root <none> and password file <none>
I20260809 03:29:17.340929 28944 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:17.340991 28944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:17.341148 28944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:17.343066 28944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "f5d3845244604caa89a0a52977f90ee7"
format_stamp: "Formatted at 2026-08-09 03:29:17 on dist-test-slave-65mx"
server_key: "3984a256594c86454bc0d2475f4aec90"
server_key_iv: "f043b56909d2a8a0aadb17fc910b45ee"
server_key_version: "encryptionkey@0"
I20260809 03:29:17.343567 28944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "f5d3845244604caa89a0a52977f90ee7"
format_stamp: "Formatted at 2026-08-09 03:29:17 on dist-test-slave-65mx"
server_key: "3984a256594c86454bc0d2475f4aec90"
server_key_iv: "f043b56909d2a8a0aadb17fc910b45ee"
server_key_version: "encryptionkey@0"
I20260809 03:29:17.347479 28944 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:29:17.349880 28962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:17.351081 28944 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:29:17.351226 28944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "f5d3845244604caa89a0a52977f90ee7"
format_stamp: "Formatted at 2026-08-09 03:29:17 on dist-test-slave-65mx"
server_key: "3984a256594c86454bc0d2475f4aec90"
server_key_iv: "f043b56909d2a8a0aadb17fc910b45ee"
server_key_version: "encryptionkey@0"
I20260809 03:29:17.351342 28944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:17.366698 28944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:17.369480 28944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:17.369683 28944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:17.376904 28944 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:41285
I20260809 03:29:17.376928 29018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:41285 every 8 connection(s)
I20260809 03:29:17.377823 28944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:17.380414 29019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:17.380779 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 28944
I20260809 03:29:17.380921 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:17.381256 9403 external_mini_cluster.cc:1428] Setting key 13ae887c7366ac6f61eaf86d7560c6ba
I20260809 03:29:17.387017 29019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7: Bootstrap starting.
I20260809 03:29:17.389366 29019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7: Neither blocks nor log segments found. Creating new log.
Aug 09 03:29:17 dist-test-slave-65mx krb5kdc[28930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:17.390165 29019 log.cc:824] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:17.392521 29019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7: No bootstrap required, opened a new log
I20260809 03:29:17.395608 29023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:17.382709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45732 (local address 127.9.46.254:41285)
0809 03:29:17.383147 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:17.383160 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:17.383197 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:29:17.383870 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:17.383885 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:17.384192 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:17.384517 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:17.385378 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:17.386363 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:17.387271 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:17.387730 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:17.387893 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:17.390467 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:17.390473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:17.390485 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:17.390512 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:17.392460 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:17.392960 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:17.392966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:17.392971 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:17.393104 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:17.393376 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:17.393379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:17.393381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:17.393878 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:17.394074 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:17.394280 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 03:29:17.394572 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":145,"threads_started":1}
I20260809 03:29:17.397122 29019 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d3845244604caa89a0a52977f90ee7" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41285 } }
I20260809 03:29:17.397413 29019 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:17.397476 29019 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5d3845244604caa89a0a52977f90ee7, State: Initialized, Role: FOLLOWER
I20260809 03:29:17.397969 29019 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d3845244604caa89a0a52977f90ee7" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41285 } }
I20260809 03:29:17.398138 29019 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:17.398231 29019 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:17.398329 29019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:17.399255 29019 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d3845244604caa89a0a52977f90ee7" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41285 } }
I20260809 03:29:17.399596 29019 leader_election.cc:302] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5d3845244604caa89a0a52977f90ee7; no voters:
I20260809 03:29:17.399907 29019 leader_election.cc:288] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:17.400111 29025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:17.400370 29025 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [term 1 LEADER]: Becoming Leader. State: Replica: f5d3845244604caa89a0a52977f90ee7, State: Running, Role: LEADER
I20260809 03:29:17.400709 29025 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d3845244604caa89a0a52977f90ee7" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41285 } }
I20260809 03:29:17.401194 29019 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:17.402416 29026 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5d3845244604caa89a0a52977f90ee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d3845244604caa89a0a52977f90ee7" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41285 } } }
I20260809 03:29:17.402473 29027 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5d3845244604caa89a0a52977f90ee7. Latest consensus state: current_term: 1 leader_uuid: "f5d3845244604caa89a0a52977f90ee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5d3845244604caa89a0a52977f90ee7" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 41285 } } }
I20260809 03:29:17.402565 29026 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:17.402570 29027 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:17.403031 29034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:17.406073 29034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:17.415566 29034 catalog_manager.cc:1347] Generated new cluster ID: 19bb160d2a904d7fbab3aef720217dc2
I20260809 03:29:17.415644 29034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:17.435323 29034 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:17.436666 29034 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:17.453492 29034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7: Generated new TSK 0
I20260809 03:29:17.454381 29034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:29:17.493080 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41285
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:17.605618 29050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:17.605912 29050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:17.605989 29050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:17.609954 29050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:17.610046 29050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:17.610141 29050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:29:17.615429 29050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:17.616467 29050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:17.617513 29050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:17.626984 29060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:17.627028 29057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:17.627049 29058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:17.627149 29050 server_base.cc:1034] running on GCE node
I20260809 03:29:17.627658 29050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:17.628362 29050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:17.629559 29050 hybrid_clock.cc:648] HybridClock initialized: now 1786246157629548 us; error 33 us; skew 500 ppm
Aug 09 03:29:17 dist-test-slave-65mx krb5kdc[28930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246157, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:17.636149 29050 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:29:17.637133 29050 webserver.cc:460] Webserver started at http://127.9.46.193:37333/ using document root <none> and password file <none>
I20260809 03:29:17.637722 29050 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:17.637785 29050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:17.637997 29050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:17.640044 29050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "fcb2c84375344452b7dea971fbe10dcd"
format_stamp: "Formatted at 2026-08-09 03:29:17 on dist-test-slave-65mx"
server_key: "61267ea7eab9ee31ae7bb9c8c1aaec6c"
server_key_iv: "affce384c19122e1329989ff48b1483c"
server_key_version: "encryptionkey@0"
I20260809 03:29:17.640712 29050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "fcb2c84375344452b7dea971fbe10dcd"
format_stamp: "Formatted at 2026-08-09 03:29:17 on dist-test-slave-65mx"
server_key: "61267ea7eab9ee31ae7bb9c8c1aaec6c"
server_key_iv: "affce384c19122e1329989ff48b1483c"
server_key_version: "encryptionkey@0"
I20260809 03:29:17.644310 29050 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 03:29:17.646744 29068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:17.647995 29050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:17.648150 29050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "fcb2c84375344452b7dea971fbe10dcd"
format_stamp: "Formatted at 2026-08-09 03:29:17 on dist-test-slave-65mx"
server_key: "61267ea7eab9ee31ae7bb9c8c1aaec6c"
server_key_iv: "affce384c19122e1329989ff48b1483c"
server_key_version: "encryptionkey@0"
I20260809 03:29:17.648267 29050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:17.666890 29050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:17.670256 29050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:17.670467 29050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:17.671180 29050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:17.672005 29050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:17.672076 29050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:17.672143 29050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:17.672191 29050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:17.680886 29050 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:44771
I20260809 03:29:17.680933 29188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:44771 every 8 connection(s)
I20260809 03:29:17.681797 29050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
Aug 09 03:29:17 dist-test-slave-65mx krb5kdc[28930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246157, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:17.690850 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 29050
I20260809 03:29:17.690989 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:29:17.691289 9403 external_mini_cluster.cc:1428] Setting key 4b0c548dc093c41b845193e2eb80c646
I20260809 03:29:17.695608 29023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:17.683730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:42259 (local address 127.9.46.254:41285)
0809 03:29:17.683914 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:17.683919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:17.684994 (+ 1075us) server_negotiation.cc:407] Connection header received
0809 03:29:17.685789 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:17.685793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:17.685841 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:17.685919 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:17.687016 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:17.687590 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:17.688691 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:17.688932 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:17.689019 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:17.691500 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:17.691503 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:17.691506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:17.691537 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:17.693284 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:17.693791 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:17.693797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:17.693801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:17.693895 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:17.694338 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:17.694345 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:17.694349 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:17.694599 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:17.694713 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:17.695270 (+ 557us) server_negotiation.cc:299] Negotiation successful
0809 03:29:17.695411 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 03:29:17.696422 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:17.684140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41285 (local address 127.9.46.193:42259)
0809 03:29:17.684853 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:17.684896 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:17.685611 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:17.686076 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:17.686082 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:17.686415 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:17.686877 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:17.686890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:17.687712 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:17.687715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:17.688584 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:17.688593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:17.689063 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:17.689065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:17.689197 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:17.689750 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:17.689765 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:17.691315 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:17.693417 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:17.693421 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:17.693431 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:17.693692 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:17.694060 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:17.694063 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:17.694069 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:17.694224 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:17.694710 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:17.694716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:17.694951 (+ 235us) client_negotiation.cc:769] Sending connection context
0809 03:29:17.695218 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 03:29:17.695552 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":122,"threads_started":1}
I20260809 03:29:17.697851 29189 heartbeater.cc:347] Connected to a master server at 127.9.46.254:41285
I20260809 03:29:17.698269 29189 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:17.698789 29189 heartbeater.cc:511] Master 127.9.46.254:41285 requested a full tablet report, sending...
I20260809 03:29:17.700529 28981 ts_manager.cc:194] Registered new tserver with Master: fcb2c84375344452b7dea971fbe10dcd (127.9.46.193:44771)
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
I20260809 03:29:17.702310 28981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:42259
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:29:17.721880 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41285
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:17.836112 29195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:17.836370 29195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:17.836413 29195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:17.840605 29195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:17.840670 29195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:17.840766 29195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:29:17.846022 29195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:17.847069 29195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:17.848129 29195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:17.857821 29200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:17.857810 29203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:17.857844 29201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:17.857908 29195 server_base.cc:1034] running on GCE node
I20260809 03:29:17.858455 29195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:17.859133 29195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:17.860332 29195 hybrid_clock.cc:648] HybridClock initialized: now 1786246157860308 us; error 41 us; skew 500 ppm
Aug 09 03:29:17 dist-test-slave-65mx krb5kdc[28930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246157, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:17.863293 29195 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:29:17.864171 29195 webserver.cc:460] Webserver started at http://127.9.46.194:42845/ using document root <none> and password file <none>
I20260809 03:29:17.864801 29195 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:17.864893 29195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:17.865134 29195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:17.867204 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "2accfb76d58e4ab28ce633a643edec27"
format_stamp: "Formatted at 2026-08-09 03:29:17 on dist-test-slave-65mx"
server_key: "e70a1dc0388653823bc46393c13c4db9"
server_key_iv: "8d6e18c1223aa2f8b782402f1c126b32"
server_key_version: "encryptionkey@0"
I20260809 03:29:17.867770 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "2accfb76d58e4ab28ce633a643edec27"
format_stamp: "Formatted at 2026-08-09 03:29:17 on dist-test-slave-65mx"
server_key: "e70a1dc0388653823bc46393c13c4db9"
server_key_iv: "8d6e18c1223aa2f8b782402f1c126b32"
server_key_version: "encryptionkey@0"
I20260809 03:29:17.871819 29195 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:29:17.874339 29210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:17.875594 29195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 03:29:17.875758 29195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "2accfb76d58e4ab28ce633a643edec27"
format_stamp: "Formatted at 2026-08-09 03:29:17 on dist-test-slave-65mx"
server_key: "e70a1dc0388653823bc46393c13c4db9"
server_key_iv: "8d6e18c1223aa2f8b782402f1c126b32"
server_key_version: "encryptionkey@0"
I20260809 03:29:17.875888 29195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:17.896906 29195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:17.900763 29195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:17.901010 29195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:17.901749 29195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:17.902925 29195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:17.903016 29195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:17.903093 29195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:17.903152 29195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:17.912778 29195 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:36527
I20260809 03:29:17.912806 29329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:36527 every 8 connection(s)
I20260809 03:29:17.913722 29195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:29:17.919329 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 29195
I20260809 03:29:17.919461 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:29:17.919845 9403 external_mini_cluster.cc:1428] Setting key cd2037ea12ac79a811ee49b9eb166793
Aug 09 03:29:17 dist-test-slave-65mx krb5kdc[28930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246157, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:17.928426 29023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:17.915675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:40963 (local address 127.9.46.254:41285)
0809 03:29:17.915811 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:17.915814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:17.916761 (+ 947us) server_negotiation.cc:407] Connection header received
0809 03:29:17.917663 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:17.917666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:17.917721 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:17.917805 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:17.919040 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:17.919958 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:17.921584 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:17.921811 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:17.921900 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:17.924863 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:17.924867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:17.924870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:17.924898 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:17.926274 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:17.926858 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:17.926862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:17.926864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:17.926949 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:17.927328 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:17.927331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:17.927333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:17.927520 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:17.927627 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:17.928130 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 03:29:17.928260 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:29:17.929306 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:17.916088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41285 (local address 127.9.46.194:40963)
0809 03:29:17.916604 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:17.916648 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:17.917431 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:17.917966 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:17.917973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:17.918336 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:17.918813 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:17.918863 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:17.920108 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:17.920112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:17.921394 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:17.921405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:17.921984 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:17.921989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:17.922246 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:17.922891 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:17.922918 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:17.924690 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:17.926399 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:17.926405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:17.926423 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:17.926742 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:17.927041 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:17.927044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:17.927046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:17.927220 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:17.927626 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:17.927634 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:17.927883 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 03:29:17.928070 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 03:29:17.928415 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:29:17.930822 29330 heartbeater.cc:347] Connected to a master server at 127.9.46.254:41285
I20260809 03:29:17.931074 29330 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:17.931553 29330 heartbeater.cc:511] Master 127.9.46.254:41285 requested a full tablet report, sending...
I20260809 03:29:17.932641 28981 ts_manager.cc:194] Registered new tserver with Master: 2accfb76d58e4ab28ce633a643edec27 (127.9.46.194:36527)
I20260809 03:29:17.933290 28981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:40963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:29:17.951357 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41285
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:33363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:18.062388 29336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:18.062644 29336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:18.062686 29336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:18.066648 29336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:18.066708 29336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:18.066766 29336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:29:18.071957 29336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:33363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:41285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:18.073019 29336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:18.073956 29336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:18.083425 29344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:18.083447 29342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:18.083454 29341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:18.083783 29336 server_base.cc:1034] running on GCE node
I20260809 03:29:18.084228 29336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:18.084923 29336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:18.086086 29336 hybrid_clock.cc:648] HybridClock initialized: now 1786246158086068 us; error 33 us; skew 500 ppm
Aug 09 03:29:18 dist-test-slave-65mx krb5kdc[28930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246158, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:18.088840 29336 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:29:18.089668 29336 webserver.cc:460] Webserver started at http://127.9.46.195:45653/ using document root <none> and password file <none>
I20260809 03:29:18.090183 29336 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:18.090241 29336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:18.090400 29336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:18.092341 29336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "989a701eecb64b9f816875aa4d436c46"
format_stamp: "Formatted at 2026-08-09 03:29:18 on dist-test-slave-65mx"
server_key: "ab9420a800e59ff25b8679990fb5576d"
server_key_iv: "fbf33452b05f3d6840f11a7b3252f2fd"
server_key_version: "encryptionkey@0"
I20260809 03:29:18.092921 29336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "989a701eecb64b9f816875aa4d436c46"
format_stamp: "Formatted at 2026-08-09 03:29:18 on dist-test-slave-65mx"
server_key: "ab9420a800e59ff25b8679990fb5576d"
server_key_iv: "fbf33452b05f3d6840f11a7b3252f2fd"
server_key_version: "encryptionkey@0"
I20260809 03:29:18.097003 29336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:29:18.099534 29351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:18.100883 29336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:18.101012 29336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "989a701eecb64b9f816875aa4d436c46"
format_stamp: "Formatted at 2026-08-09 03:29:18 on dist-test-slave-65mx"
server_key: "ab9420a800e59ff25b8679990fb5576d"
server_key_iv: "fbf33452b05f3d6840f11a7b3252f2fd"
server_key_version: "encryptionkey@0"
I20260809 03:29:18.101137 29336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:18.123682 29336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:18.126802 29336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:18.127027 29336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:18.127664 29336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:18.128648 29336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:18.128716 29336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:18.128788 29336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:18.128840 29336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:18.138343 29336 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:42311
I20260809 03:29:18.138366 29470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:42311 every 8 connection(s)
I20260809 03:29:18.139299 29336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
Aug 09 03:29:18 dist-test-slave-65mx krb5kdc[28930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246158, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:18.148809 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 29336
I20260809 03:29:18.148957 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:29:18.149320 9403 external_mini_cluster.cc:1428] Setting key 81be0a822acfb5d871ac53b3259f7d47
I20260809 03:29:18.152925 29023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.141209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:57783 (local address 127.9.46.254:41285)
0809 03:29:18.141370 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:18.141373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:18.142223 (+ 850us) server_negotiation.cc:407] Connection header received
0809 03:29:18.143121 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:18.143124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:18.143202 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:18.143306 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:18.144433 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.145022 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.146143 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.146361 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.146472 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.148997 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:18.149005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:18.149008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:18.149031 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:18.150641 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:18.151278 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:18.151281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:18.151282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:18.151362 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:18.151831 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:18.151833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:18.151834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:18.152080 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:18.152168 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:18.152700 (+ 532us) server_negotiation.cc:299] Negotiation successful
0809 03:29:18.152808 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 03:29:18.153784 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.141614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:41285 (local address 127.9.46.195:57783)
0809 03:29:18.142069 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:18.142110 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:18.142907 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:18.143473 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:18.143479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:18.143836 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:18.144279 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.144291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.145150 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.145155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.146003 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.146012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.146523 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.146528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.146694 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.147299 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:18.147315 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:18.148827 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:18.150771 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:18.150777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:18.150800 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:18.151181 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:18.151523 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:18.151526 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:18.151529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:18.151735 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:18.152186 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:18.152191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:18.152419 (+ 228us) client_negotiation.cc:769] Sending connection context
0809 03:29:18.152661 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 03:29:18.152926 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":122,"threads_started":1}
I20260809 03:29:18.154834 29471 heartbeater.cc:347] Connected to a master server at 127.9.46.254:41285
I20260809 03:29:18.155068 29471 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:18.155540 29471 heartbeater.cc:511] Master 127.9.46.254:41285 requested a full tablet report, sending...
I20260809 03:29:18.156807 28981 ts_manager.cc:194] Registered new tserver with Master: 989a701eecb64b9f816875aa4d436c46 (127.9.46.195:42311)
I20260809 03:29:18.157379 28981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:57783
I20260809 03:29:18.163317 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:29:18.172701 29023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.165356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45748 (local address 127.9.46.254:41285)
0809 03:29:18.165491 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:18.165495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:18.165691 (+ 196us) server_negotiation.cc:407] Connection header received
0809 03:29:18.165785 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:18.165788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:18.165826 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:18.165890 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:18.166510 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.167003 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.167806 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.168021 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.168094 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.169521 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:18.169524 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:18.169526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:18.169548 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:18.170984 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:18.171471 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:18.171474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:18.171475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:18.171547 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:18.171939 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:18.171942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:18.171943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:18.172141 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:18.172259 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:18.172446 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 03:29:18.172519 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:29:18.175467 28981 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:29:18.178493 28981 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:29:18.193025 29483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.186714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:36527 (local address 127.0.0.1:42774)
0809 03:29:18.187210 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:18.187260 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:18.187392 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:18.188786 (+ 1394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:18.188791 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:18.188813 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:18.188891 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.188897 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.190450 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.190453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.191467 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.191481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.192597 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.192600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.192755 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.192776 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:29:18.192836 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 03:29:18.192899 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":117,"thread_start_us":94,"threads_started":1}
I20260809 03:29:18.193078 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.186872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.9.46.194:36527)
0809 03:29:18.188276 (+ 1404us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:18.188289 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:18.188313 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:29:18.188398 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:18.188404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:18.188621 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:18.189033 (+ 412us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:18.189187 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.190256 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.191588 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.192420 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.192652 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.192802 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:18.192859 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 03:29:18.192952 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1297,"thread_start_us":105,"threads_started":1}
I20260809 03:29:18.193527 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.187859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45654 (local address 127.9.46.193:44771)
0809 03:29:18.188216 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:18.188233 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:18.188254 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:29:18.188331 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:18.188337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:18.188504 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:18.188675 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:18.188891 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.189614 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.191690 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.192461 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.193085 (+ 624us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.193257 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:18.193308 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 03:29:18.193390 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":71,"threads_started":1}
I20260809 03:29:18.193590 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.187375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.9.46.195:42311)
0809 03:29:18.187729 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:18.187743 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:18.187919 (+ 176us) server_negotiation.cc:407] Connection header received
0809 03:29:18.188036 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:18.188041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:18.188254 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:18.188414 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:18.188749 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.189916 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.191706 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.192249 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.192377 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.192525 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:18.193346 (+ 821us) server_negotiation.cc:299] Negotiation successful
0809 03:29:18.193452 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":148,"threads_started":1}
I20260809 03:29:18.193037 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.187410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44771 (local address 127.0.0.1:45654)
0809 03:29:18.188138 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:18.188152 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:18.188236 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:18.188650 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:18.188653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:18.188673 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:18.188772 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.188779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.189750 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.189755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.191540 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.191554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.192600 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.192603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.192704 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.192727 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:29:18.192790 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 03:29:18.192868 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":356,"thread_start_us":109,"threads_started":1}
I20260809 03:29:18.193027 29487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.186885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:42311 (local address 127.0.0.1:37608)
0809 03:29:18.187804 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:18.187820 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:18.187919 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:18.188413 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:18.188418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:18.188440 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:18.188602 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.188609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.190054 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.190057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.191581 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.191590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.192379 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.192383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.192486 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.192521 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 03:29:18.192673 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 03:29:18.192794 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":171,"thread_start_us":102,"threads_started":1}
I20260809 03:29:18.196229 29401 tablet_service.cc:1467] Processing CreateTablet for tablet a4e1d7dcb87040aba7e84c738692f3f2 (DEFAULT_TABLE table=test-table [id=a41a353a85a24df4b104eda5c8c22ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:18.196393 29119 tablet_service.cc:1467] Processing CreateTablet for tablet a4e1d7dcb87040aba7e84c738692f3f2 (DEFAULT_TABLE table=test-table [id=a41a353a85a24df4b104eda5c8c22ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:18.196393 29260 tablet_service.cc:1467] Processing CreateTablet for tablet a4e1d7dcb87040aba7e84c738692f3f2 (DEFAULT_TABLE table=test-table [id=a41a353a85a24df4b104eda5c8c22ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:18.197410 29401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e1d7dcb87040aba7e84c738692f3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:18.197429 29119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e1d7dcb87040aba7e84c738692f3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:18.197430 29260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e1d7dcb87040aba7e84c738692f3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:18.203910 29489 tablet_bootstrap.cc:492] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: Bootstrap starting.
I20260809 03:29:18.204761 29491 tablet_bootstrap.cc:492] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: Bootstrap starting.
I20260809 03:29:18.206578 29489 tablet_bootstrap.cc:654] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:18.207015 29491 tablet_bootstrap.cc:654] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:18.207660 29489 log.cc:824] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:18.207664 29491 log.cc:824] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:18.208367 29490 tablet_bootstrap.cc:492] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd: Bootstrap starting.
I20260809 03:29:18.209517 29489 tablet_bootstrap.cc:492] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: No bootstrap required, opened a new log
I20260809 03:29:18.209728 29489 ts_tablet_manager.cc:1397] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:29:18.210151 29490 tablet_bootstrap.cc:654] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:18.210816 29490 log.cc:824] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:18.212476 29491 tablet_bootstrap.cc:492] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: No bootstrap required, opened a new log
I20260809 03:29:18.212623 29490 tablet_bootstrap.cc:492] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd: No bootstrap required, opened a new log
I20260809 03:29:18.212723 29491 ts_tablet_manager.cc:1397] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 03:29:18.212848 29490 ts_tablet_manager.cc:1397] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:29:18.213515 29489 raft_consensus.cc:348] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 } } peers { permanent_uuid: "989a701eecb64b9f816875aa4d436c46" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42311 } } peers { permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 } }
I20260809 03:29:18.213810 29489 raft_consensus.cc:374] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:18.213869 29489 raft_consensus.cc:729] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989a701eecb64b9f816875aa4d436c46, State: Initialized, Role: FOLLOWER
I20260809 03:29:18.214334 29489 consensus_queue.cc:260] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 } } peers { permanent_uuid: "989a701eecb64b9f816875aa4d436c46" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42311 } } peers { permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 } }
I20260809 03:29:18.215045 29471 heartbeater.cc:503] Master 127.9.46.254:41285 was elected leader, sending a full tablet report...
I20260809 03:29:18.215392 29489 ts_tablet_manager.cc:1428] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:29:18.216534 29491 raft_consensus.cc:348] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 } } peers { permanent_uuid: "989a701eecb64b9f816875aa4d436c46" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42311 } } peers { permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 } }
I20260809 03:29:18.216861 29491 raft_consensus.cc:374] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:18.216920 29491 raft_consensus.cc:729] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2accfb76d58e4ab28ce633a643edec27, State: Initialized, Role: FOLLOWER
I20260809 03:29:18.217036 29490 raft_consensus.cc:348] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 } } peers { permanent_uuid: "989a701eecb64b9f816875aa4d436c46" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42311 } } peers { permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 } }
I20260809 03:29:18.217268 29490 raft_consensus.cc:374] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:18.217329 29490 raft_consensus.cc:729] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb2c84375344452b7dea971fbe10dcd, State: Initialized, Role: FOLLOWER
I20260809 03:29:18.217360 29491 consensus_queue.cc:260] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 } } peers { permanent_uuid: "989a701eecb64b9f816875aa4d436c46" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42311 } } peers { permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 } }
I20260809 03:29:18.217799 29490 consensus_queue.cc:260] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 } } peers { permanent_uuid: "989a701eecb64b9f816875aa4d436c46" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42311 } } peers { permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 } }
I20260809 03:29:18.218432 29491 ts_tablet_manager.cc:1428] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260809 03:29:18.218699 29490 ts_tablet_manager.cc:1428] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:29:18.218887 29330 heartbeater.cc:503] Master 127.9.46.254:41285 was elected leader, sending a full tablet report...
I20260809 03:29:18.218981 29189 heartbeater.cc:503] Master 127.9.46.254:41285 was elected leader, sending a full tablet report...
I20260809 03:29:18.280759 29495 raft_consensus.cc:482] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:18.280997 29495 raft_consensus.cc:504] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 } } peers { permanent_uuid: "989a701eecb64b9f816875aa4d436c46" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42311 } } peers { permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 } }
I20260809 03:29:18.282157 29495 leader_election.cc:288] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2accfb76d58e4ab28ce633a643edec27 (127.9.46.194:36527), fcb2c84375344452b7dea971fbe10dcd (127.9.46.193:44771)
I20260809 03:29:18.286154 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.282394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:36527 (local address 127.9.46.195:40275)
0809 03:29:18.282525 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:18.282542 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:18.282651 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:18.282930 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:18.282933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:18.282953 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:18.283102 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.283109 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.283983 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.283986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.284892 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.284901 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.285726 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.285730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.285840 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.285859 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 03:29:18.285949 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:29:18.286022 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 03:29:18.286162 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.282488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:40275 (local address 127.9.46.194:36527)
0809 03:29:18.282621 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:18.282625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:18.282642 (+ 17us) server_negotiation.cc:407] Connection header received
0809 03:29:18.282762 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:18.282766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:18.282823 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:18.282908 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:18.283238 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.283875 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.285016 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.285607 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.285726 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.285780 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:18.285964 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 03:29:18.286031 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 03:29:18.286476 29498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.282413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44771 (local address 127.9.46.195:43119)
0809 03:29:18.282929 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:18.282942 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:18.283035 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:18.283328 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:18.283331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:18.283348 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:18.283454 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.283460 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.284248 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.284250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.285171 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.285181 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.286031 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.286035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.286138 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.286155 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 03:29:18.286255 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 03:29:18.286345 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":117,"thread_start_us":134,"threads_started":1}
I20260809 03:29:18.286767 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.282566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:43119 (local address 127.9.46.193:44771)
0809 03:29:18.282731 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:18.282735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:18.283031 (+ 296us) server_negotiation.cc:407] Connection header received
0809 03:29:18.283146 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:18.283151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:18.283214 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:18.283291 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:18.283571 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.284135 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.285356 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.285860 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.285983 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.286035 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:18.286545 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 03:29:18.286638 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:29:18.287026 29280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e1d7dcb87040aba7e84c738692f3f2" candidate_uuid: "989a701eecb64b9f816875aa4d436c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2accfb76d58e4ab28ce633a643edec27" is_pre_election: true
I20260809 03:29:18.287282 29140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e1d7dcb87040aba7e84c738692f3f2" candidate_uuid: "989a701eecb64b9f816875aa4d436c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb2c84375344452b7dea971fbe10dcd" is_pre_election: true
I20260809 03:29:18.287358 29280 raft_consensus.cc:2454] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 989a701eecb64b9f816875aa4d436c46 in term 0.
I20260809 03:29:18.287544 29140 raft_consensus.cc:2454] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 989a701eecb64b9f816875aa4d436c46 in term 0.
I20260809 03:29:18.287870 29352 leader_election.cc:302] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2accfb76d58e4ab28ce633a643edec27, 989a701eecb64b9f816875aa4d436c46; no voters:
I20260809 03:29:18.288172 29495 raft_consensus.cc:2781] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:29:18.288260 29495 raft_consensus.cc:482] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:29:18.288293 29495 raft_consensus.cc:3037] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:18.289376 29495 raft_consensus.cc:504] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 } } peers { permanent_uuid: "989a701eecb64b9f816875aa4d436c46" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42311 } } peers { permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 } }
I20260809 03:29:18.289726 29495 leader_election.cc:288] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [CANDIDATE]: Term 1 election: Requested vote from peers 2accfb76d58e4ab28ce633a643edec27 (127.9.46.194:36527), fcb2c84375344452b7dea971fbe10dcd (127.9.46.193:44771)
I20260809 03:29:18.290139 29140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e1d7dcb87040aba7e84c738692f3f2" candidate_uuid: "989a701eecb64b9f816875aa4d436c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb2c84375344452b7dea971fbe10dcd"
I20260809 03:29:18.290112 29280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e1d7dcb87040aba7e84c738692f3f2" candidate_uuid: "989a701eecb64b9f816875aa4d436c46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2accfb76d58e4ab28ce633a643edec27"
I20260809 03:29:18.290251 29140 raft_consensus.cc:3037] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:18.290251 29280 raft_consensus.cc:3037] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:18.291491 29280 raft_consensus.cc:2454] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 989a701eecb64b9f816875aa4d436c46 in term 1.
I20260809 03:29:18.291491 29140 raft_consensus.cc:2454] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 989a701eecb64b9f816875aa4d436c46 in term 1.
I20260809 03:29:18.291805 29352 leader_election.cc:302] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2accfb76d58e4ab28ce633a643edec27, 989a701eecb64b9f816875aa4d436c46, fcb2c84375344452b7dea971fbe10dcd; no voters:
I20260809 03:29:18.291947 29495 raft_consensus.cc:2781] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:18.292162 29495 raft_consensus.cc:686] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 1 LEADER]: Becoming Leader. State: Replica: 989a701eecb64b9f816875aa4d436c46, State: Running, Role: LEADER
I20260809 03:29:18.292498 29495 consensus_queue.cc:237] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 } } peers { permanent_uuid: "989a701eecb64b9f816875aa4d436c46" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42311 } } peers { permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 } }
I20260809 03:29:18.295524 28980 catalog_manager.cc:5495] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 989a701eecb64b9f816875aa4d436c46 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "989a701eecb64b9f816875aa4d436c46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "989a701eecb64b9f816875aa4d436c46" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 } health_report { overall_health: UNKNOWN } } }
I20260809 03:29:18.319211 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.315346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37612 (local address 127.9.46.195:42311)
0809 03:29:18.315512 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:18.315517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:18.315624 (+ 107us) server_negotiation.cc:407] Connection header received
0809 03:29:18.315832 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:18.315834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:18.315891 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:18.316005 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:18.316278 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.316850 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.317726 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.317939 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.318044 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.318363 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:18.318758 (+ 395us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:18.318919 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:18.319008 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 03:29:18.319048 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 03:29:18.324954 29140 raft_consensus.cc:1261] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd [term 1 FOLLOWER]: Refusing update from remote peer 989a701eecb64b9f816875aa4d436c46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:18.324954 29280 raft_consensus.cc:1261] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27 [term 1 FOLLOWER]: Refusing update from remote peer 989a701eecb64b9f816875aa4d436c46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:18.325640 29499 consensus_queue.cc:1035] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2accfb76d58e4ab28ce633a643edec27" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 36527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:18.325838 29495 consensus_queue.cc:1035] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb2c84375344452b7dea971fbe10dcd" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:18.334097 29503 mvcc.cc:205] Tried to move back new op lower bound from 7316464264491974656 to 7316464264368803840. Current Snapshot: MvccSnapshot[applied={T|T < 7316464264491974656}]
I20260809 03:29:18.341518 28980 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:18.341683 28980 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:18.344012 28980 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5d3845244604caa89a0a52977f90ee7: Sending DeleteTablet for 3 replicas of tablet a4e1d7dcb87040aba7e84c738692f3f2
I20260809 03:29:18.344795 29401 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e1d7dcb87040aba7e84c738692f3f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:18 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:37608
I20260809 03:29:18.344796 29119 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e1d7dcb87040aba7e84c738692f3f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:18 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:45654
I20260809 03:29:18.344815 29260 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e1d7dcb87040aba7e84c738692f3f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:18 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:42774
I20260809 03:29:18.345338 29513 tablet_replica.cc:330] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd: stopping tablet replica
I20260809 03:29:18.345510 29515 tablet_replica.cc:330] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: stopping tablet replica
I20260809 03:29:18.345664 29513 raft_consensus.cc:2229] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:29:18.345896 29515 raft_consensus.cc:2229] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:29:18.346004 29513 raft_consensus.cc:2258] T a4e1d7dcb87040aba7e84c738692f3f2 P fcb2c84375344452b7dea971fbe10dcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:18.346241 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 29050
I20260809 03:29:18.347757 29514 tablet_replica.cc:330] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: stopping tablet replica
I20260809 03:29:18.348122 29514 raft_consensus.cc:2229] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:29:18.348519 29514 raft_consensus.cc:2258] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:18.350257 29514 ts_tablet_manager.cc:1905] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:29:18.346283 29515 raft_consensus.cc:2258] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:18.352783 29515 ts_tablet_manager.cc:1905] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:29:18.353919 29514 ts_tablet_manager.cc:1918] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:29:18.354029 29514 log.cc:1196] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/a4e1d7dcb87040aba7e84c738692f3f2
I20260809 03:29:18.354390 29514 ts_tablet_manager.cc:1939] T a4e1d7dcb87040aba7e84c738692f3f2 P 989a701eecb64b9f816875aa4d436c46: Deleting consensus metadata
I20260809 03:29:18.355767 28967 catalog_manager.cc:4841] TS 989a701eecb64b9f816875aa4d436c46 (127.9.46.195:42311): tablet a4e1d7dcb87040aba7e84c738692f3f2 (table test-table [id=a41a353a85a24df4b104eda5c8c22ba4]) successfully deleted
I20260809 03:29:18.355854 29515 ts_tablet_manager.cc:1918] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:29:18.356067 29515 log.cc:1196] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/a4e1d7dcb87040aba7e84c738692f3f2
I20260809 03:29:18.356415 29515 ts_tablet_manager.cc:1939] T a4e1d7dcb87040aba7e84c738692f3f2 P 2accfb76d58e4ab28ce633a643edec27: Deleting consensus metadata
W20260809 03:29:18.357276 28965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:44771 (error 108)
I20260809 03:29:18.357613 28963 catalog_manager.cc:4841] TS 2accfb76d58e4ab28ce633a643edec27 (127.9.46.194:36527): tablet a4e1d7dcb87040aba7e84c738692f3f2 (table test-table [id=a41a353a85a24df4b104eda5c8c22ba4]) successfully deleted
I20260809 03:29:18.358196 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 29195
I20260809 03:29:18.358503 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.358196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44771 (local address 127.0.0.1:45664)
0809 03:29:18.358334 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:18.358408 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 03:29:18.359702 28965 catalog_manager.cc:4584] TS fcb2c84375344452b7dea971fbe10dcd (127.9.46.193:44771): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4e1d7dcb87040aba7e84c738692f3f2: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44771: connect: Connection refused (error 111)
I20260809 03:29:18.368165 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 29336
I20260809 03:29:18.376025 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 28944
2026-08-09T03:29:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2385 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T03:29:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:18Z Disabled control of system clock
I20260809 03:29:18.410539 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39239
--webserver_interface=127.9.46.254
--webserver_port=0
--builtin_ntp_servers=127.9.46.212:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:39239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:29:18.565968 29520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:18.566407 29520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:18.566465 29520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:18.572319 29520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:18.572477 29520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:18.572566 29520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:18.572654 29520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:18.572715 29520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:18.580328 29520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:39239
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:18.582373 29520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:18.584569 29520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:29:18.598855 29520 server_base.cc:1034] running on GCE node
W20260809 03:29:18.598723 29527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:18.598760 29526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:18.598685 29529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:18.600525 29520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:18.602485 29520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:18.603710 29520 hybrid_clock.cc:648] HybridClock initialized: now 1786246158603647 us; error 73 us; skew 500 ppm
I20260809 03:29:18.606469 29520 webserver.cc:460] Webserver started at http://127.9.46.254:40135/ using document root <none> and password file <none>
I20260809 03:29:18.607362 29520 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:18.607466 29520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:18.607815 29520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:18.611035 29520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "998d5080f51a4cca85502c56ce6532a5"
format_stamp: "Formatted at 2026-08-09 03:29:18 on dist-test-slave-65mx"
server_key: "95273258186185e8e8037e12e7493583"
server_key_iv: "6774105af0bf2eca856d10970e59c6c3"
server_key_version: "encryptionkey@0"
I20260809 03:29:18.611907 29520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "998d5080f51a4cca85502c56ce6532a5"
format_stamp: "Formatted at 2026-08-09 03:29:18 on dist-test-slave-65mx"
server_key: "95273258186185e8e8037e12e7493583"
server_key_iv: "6774105af0bf2eca856d10970e59c6c3"
server_key_version: "encryptionkey@0"
I20260809 03:29:18.621397 29520 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.010s sys 0.001s
I20260809 03:29:18.626981 29535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:18.629652 29520 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260809 03:29:18.629858 29520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "998d5080f51a4cca85502c56ce6532a5"
format_stamp: "Formatted at 2026-08-09 03:29:18 on dist-test-slave-65mx"
server_key: "95273258186185e8e8037e12e7493583"
server_key_iv: "6774105af0bf2eca856d10970e59c6c3"
server_key_version: "encryptionkey@0"
I20260809 03:29:18.630048 29520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:18.669524 29520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:18.670220 29520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:18.670378 29520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:18.677258 29520 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:39239
I20260809 03:29:18.677270 29589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:39239 every 8 connection(s)
I20260809 03:29:18.678210 29520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:18.679714 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 29520
I20260809 03:29:18.679859 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:18.680184 9403 external_mini_cluster.cc:1428] Setting key bf0d1872324bafc2c2295438cd631fa9
I20260809 03:29:18.682754 29590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:18.688165 29590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5: Bootstrap starting.
I20260809 03:29:18.689008 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.681453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37186 (local address 127.9.46.254:39239)
0809 03:29:18.682048 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:18.682063 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:18.682097 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:29:18.682729 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:18.682744 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:18.683159 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:18.683476 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:18.683896 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.684926 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.685792 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.686254 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.686426 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.687289 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:18.687293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:18.687304 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:18.687681 (+ 377us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:18.687735 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:18.687755 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:18.687861 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:18.687990 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 03:29:18.688255 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":122,"threads_started":1}
I20260809 03:29:18.690593 29590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:18.691471 29590 log.cc:824] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:18.693357 29590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5: No bootstrap required, opened a new log
I20260809 03:29:18.696969 29590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d5080f51a4cca85502c56ce6532a5" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39239 } }
I20260809 03:29:18.697180 29590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:18.697214 29590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998d5080f51a4cca85502c56ce6532a5, State: Initialized, Role: FOLLOWER
I20260809 03:29:18.697607 29590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d5080f51a4cca85502c56ce6532a5" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39239 } }
I20260809 03:29:18.697793 29590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:18.697865 29590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:18.697971 29590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:18.698863 29590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d5080f51a4cca85502c56ce6532a5" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39239 } }
I20260809 03:29:18.699189 29590 leader_election.cc:302] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 998d5080f51a4cca85502c56ce6532a5; no voters:
I20260809 03:29:18.699505 29590 leader_election.cc:288] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:18.699612 29595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:18.699864 29595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [term 1 LEADER]: Becoming Leader. State: Replica: 998d5080f51a4cca85502c56ce6532a5, State: Running, Role: LEADER
I20260809 03:29:18.700161 29595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d5080f51a4cca85502c56ce6532a5" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39239 } }
I20260809 03:29:18.700790 29590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:18.701956 29596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "998d5080f51a4cca85502c56ce6532a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d5080f51a4cca85502c56ce6532a5" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39239 } } }
I20260809 03:29:18.702049 29597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 998d5080f51a4cca85502c56ce6532a5. Latest consensus state: current_term: 1 leader_uuid: "998d5080f51a4cca85502c56ce6532a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998d5080f51a4cca85502c56ce6532a5" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39239 } } }
I20260809 03:29:18.702108 29596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:18.702121 29597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:18.702613 29600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:18.705310 29600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:18.710502 29600 catalog_manager.cc:1347] Generated new cluster ID: 4968662f8454494ab43efe81962f0464
I20260809 03:29:18.710567 29600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:18.726348 29600 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:18.727295 29600 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:18.737807 29600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5: Generated new TSK 0
I20260809 03:29:18.738610 29600 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:18.749647 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39239
--builtin_ntp_servers=127.9.46.212:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:29:18.865142 29617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:18.865419 29617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:18.865458 29617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:18.869601 29617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:18.869665 29617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:18.869745 29617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:29:18.874588 29617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:18.875699 29617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:18.877147 29617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:18.886653 29625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:18.886642 29622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:18.886642 29623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:18.886926 29617 server_base.cc:1034] running on GCE node
I20260809 03:29:18.887465 29617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:18.888116 29617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:18.889310 29617 hybrid_clock.cc:648] HybridClock initialized: now 1786246158889297 us; error 30 us; skew 500 ppm
I20260809 03:29:18.891242 29617 webserver.cc:460] Webserver started at http://127.9.46.193:44383/ using document root <none> and password file <none>
I20260809 03:29:18.891850 29617 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:18.891947 29617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:18.892190 29617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:18.894291 29617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "1f45a70f04244e94aa6cc1c6e827ba44"
format_stamp: "Formatted at 2026-08-09 03:29:18 on dist-test-slave-65mx"
server_key: "cebf63ed9c914faa869d4b2ac2e4aff5"
server_key_iv: "4bc10ed54b27d7ab4bc0fedfc8658446"
server_key_version: "encryptionkey@0"
I20260809 03:29:18.894855 29617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "1f45a70f04244e94aa6cc1c6e827ba44"
format_stamp: "Formatted at 2026-08-09 03:29:18 on dist-test-slave-65mx"
server_key: "cebf63ed9c914faa869d4b2ac2e4aff5"
server_key_iv: "4bc10ed54b27d7ab4bc0fedfc8658446"
server_key_version: "encryptionkey@0"
I20260809 03:29:18.898488 29617 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:29:18.900810 29633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:18.902105 29617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:18.902241 29617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "1f45a70f04244e94aa6cc1c6e827ba44"
format_stamp: "Formatted at 2026-08-09 03:29:18 on dist-test-slave-65mx"
server_key: "cebf63ed9c914faa869d4b2ac2e4aff5"
server_key_iv: "4bc10ed54b27d7ab4bc0fedfc8658446"
server_key_version: "encryptionkey@0"
I20260809 03:29:18.902346 29617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:18.914891 29617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:18.915529 29617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:18.915721 29617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:18.916275 29617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:18.917245 29617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:18.917323 29617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:18.917409 29617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:18.917455 29617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:18.926774 29617 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:39617
I20260809 03:29:18.926811 29752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:39617 every 8 connection(s)
I20260809 03:29:18.927700 29617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:29:18.936620 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.929661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:58589 (local address 127.9.46.254:39239)
0809 03:29:18.929844 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:18.929848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:18.930891 (+ 1043us) server_negotiation.cc:407] Connection header received
0809 03:29:18.932196 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:18.932199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:18.932251 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:18.932327 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:18.933213 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.933757 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.934865 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.935075 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.935150 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.936141 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:18.936144 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:18.936146 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:18.936158 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:18.936177 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:18.936184 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:18.936243 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:18.936369 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 03:29:18.936444 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 03:29:18.936990 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 29617
I20260809 03:29:18.937140 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:29:18.937470 9403 external_mini_cluster.cc:1428] Setting key e49549c7b6bb6580acb76100e8ce85df
I20260809 03:29:18.937811 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:18.930060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:39239 (local address 127.9.46.193:58589)
0809 03:29:18.930730 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:18.930771 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:18.932014 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:18.932481 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:18.932487 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:18.932584 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:18.933078 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.933090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.933870 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.933872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.934762 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:18.934771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:18.935198 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:18.935201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:18.935338 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:18.935903 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:29:18.935905 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:18.935944 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:29:18.936003 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:18.936253 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:18.936261 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:18.936265 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:29:18.936355 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 03:29:18.936620 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":118,"threads_started":1}
I20260809 03:29:18.939054 29753 heartbeater.cc:347] Connected to a master server at 127.9.46.254:39239
I20260809 03:29:18.939297 29753 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:18.939690 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39239
--builtin_ntp_servers=127.9.46.212:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:29:18.939921 29753 heartbeater.cc:511] Master 127.9.46.254:39239 requested a full tablet report, sending...
I20260809 03:29:18.941641 29552 ts_manager.cc:194] Registered new tserver with Master: 1f45a70f04244e94aa6cc1c6e827ba44 (127.9.46.193:39617)
I20260809 03:29:18.942953 29552 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.193:58589
W20260809 03:29:19.051045 29758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:19.051319 29758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:19.051360 29758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:19.055312 29758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:19.055372 29758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:19.055428 29758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:29:19.060032 29758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:19.061071 29758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:19.062393 29758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:19.071859 29766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:19.071913 29758 server_base.cc:1034] running on GCE node
W20260809 03:29:19.071852 29764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:19.071825 29763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:19.072695 29758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:19.073321 29758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:19.074512 29758 hybrid_clock.cc:648] HybridClock initialized: now 1786246159074494 us; error 28 us; skew 500 ppm
I20260809 03:29:19.076720 29758 webserver.cc:460] Webserver started at http://127.9.46.194:37951/ using document root <none> and password file <none>
I20260809 03:29:19.077337 29758 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:19.077411 29758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:19.077651 29758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:19.079873 29758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "f388dfe40f0c4926816d17f8a0c4fa35"
format_stamp: "Formatted at 2026-08-09 03:29:19 on dist-test-slave-65mx"
server_key: "e8a28d8030e90ecd94d7abfec790b8c8"
server_key_iv: "99e4c6972853f7a02f648ece33321468"
server_key_version: "encryptionkey@0"
I20260809 03:29:19.080493 29758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "f388dfe40f0c4926816d17f8a0c4fa35"
format_stamp: "Formatted at 2026-08-09 03:29:19 on dist-test-slave-65mx"
server_key: "e8a28d8030e90ecd94d7abfec790b8c8"
server_key_iv: "99e4c6972853f7a02f648ece33321468"
server_key_version: "encryptionkey@0"
I20260809 03:29:19.084458 29758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 03:29:19.087275 29773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:19.088493 29758 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 03:29:19.088668 29758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "f388dfe40f0c4926816d17f8a0c4fa35"
format_stamp: "Formatted at 2026-08-09 03:29:19 on dist-test-slave-65mx"
server_key: "e8a28d8030e90ecd94d7abfec790b8c8"
server_key_iv: "99e4c6972853f7a02f648ece33321468"
server_key_version: "encryptionkey@0"
I20260809 03:29:19.088780 29758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:19.116675 29758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:19.117697 29758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:19.117937 29758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:19.118616 29758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:19.119632 29758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:19.119683 29758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:19.119755 29758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:19.119800 29758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:19.129648 29758 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:33219
I20260809 03:29:19.129673 29895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:33219 every 8 connection(s)
I20260809 03:29:19.130600 29758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:29:19.137873 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 29758
I20260809 03:29:19.138024 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:29:19.138325 9403 external_mini_cluster.cc:1428] Setting key c288a7aa1ac324e7befd81d4edba92e2
I20260809 03:29:19.140157 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.132405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:48785 (local address 127.9.46.254:39239)
0809 03:29:19.132598 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:19.132603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:19.133726 (+ 1123us) server_negotiation.cc:407] Connection header received
0809 03:29:19.135130 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:19.135134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:19.135206 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:19.135301 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:19.136174 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.136707 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.138023 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.138341 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.138467 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.139650 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:19.139653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:19.139655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:19.139666 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:19.139686 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:19.139694 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:19.139774 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:19.139935 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 03:29:19.140012 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:29:19.140523 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39239
--builtin_ntp_servers=127.9.46.212:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:29:19.141073 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.132916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:39239 (local address 127.9.46.194:48785)
0809 03:29:19.133571 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:19.133610 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:19.134919 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:19.135473 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:19.135479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:19.135557 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:19.136024 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.136036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.136844 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.136847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.137819 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.137832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.138528 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.138535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.138713 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.139374 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:29:19.139375 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:19.139415 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:29:19.139476 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:19.139776 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:19.139787 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:19.139797 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 03:29:19.139906 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 03:29:19.140121 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":130,"threads_started":1}
I20260809 03:29:19.142166 29896 heartbeater.cc:347] Connected to a master server at 127.9.46.254:39239
I20260809 03:29:19.142385 29896 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:19.142882 29896 heartbeater.cc:511] Master 127.9.46.254:39239 requested a full tablet report, sending...
I20260809 03:29:19.143867 29552 ts_manager.cc:194] Registered new tserver with Master: f388dfe40f0c4926816d17f8a0c4fa35 (127.9.46.194:33219)
I20260809 03:29:19.144439 29552 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.194:48785
W20260809 03:29:19.255852 29901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:19.256165 29901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:19.256243 29901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:19.260375 29901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:19.260457 29901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:19.260579 29901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:29:19.265487 29901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:39239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:19.266623 29901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:19.268078 29901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:19.277547 29910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:19.277554 29907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:19.277558 29906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:19.277750 29901 server_base.cc:1034] running on GCE node
I20260809 03:29:19.278208 29901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:19.278841 29901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:19.280017 29901 hybrid_clock.cc:648] HybridClock initialized: now 1786246159280001 us; error 30 us; skew 500 ppm
I20260809 03:29:19.281803 29901 webserver.cc:460] Webserver started at http://127.9.46.195:46753/ using document root <none> and password file <none>
I20260809 03:29:19.282359 29901 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:19.282446 29901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:19.282675 29901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:19.284777 29901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "e0eb3c4549af459b94f5a0d9c5fe4551"
format_stamp: "Formatted at 2026-08-09 03:29:19 on dist-test-slave-65mx"
server_key: "e1cbdd289b8b8cc7b50aef96cc0406f4"
server_key_iv: "005ffb88e91b03b7732b3f9167724e1a"
server_key_version: "encryptionkey@0"
I20260809 03:29:19.285329 29901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "e0eb3c4549af459b94f5a0d9c5fe4551"
format_stamp: "Formatted at 2026-08-09 03:29:19 on dist-test-slave-65mx"
server_key: "e1cbdd289b8b8cc7b50aef96cc0406f4"
server_key_iv: "005ffb88e91b03b7732b3f9167724e1a"
server_key_version: "encryptionkey@0"
I20260809 03:29:19.288884 29901 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:29:19.291256 29916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:19.292466 29901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:19.292616 29901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "e0eb3c4549af459b94f5a0d9c5fe4551"
format_stamp: "Formatted at 2026-08-09 03:29:19 on dist-test-slave-65mx"
server_key: "e1cbdd289b8b8cc7b50aef96cc0406f4"
server_key_iv: "005ffb88e91b03b7732b3f9167724e1a"
server_key_version: "encryptionkey@0"
I20260809 03:29:19.292739 29901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:19.311677 29901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:19.312379 29901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:19.312629 29901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:19.313236 29901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:19.314188 29901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:19.314256 29901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:19.314327 29901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:19.314373 29901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:19.322908 29901 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:42957
I20260809 03:29:19.322932 30040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:42957 every 8 connection(s)
I20260809 03:29:19.323879 29901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:29:19.328142 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 29901
I20260809 03:29:19.328269 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:29:19.328613 9403 external_mini_cluster.cc:1428] Setting key cbe1f702b1a1a6ed9f20c5bce62e2cde
I20260809 03:29:19.333242 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.325724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:48291 (local address 127.9.46.254:39239)
0809 03:29:19.325895 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:19.325900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:19.326928 (+ 1028us) server_negotiation.cc:407] Connection header received
0809 03:29:19.328264 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:19.328268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:19.328332 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:19.328434 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:19.329538 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.330244 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.331486 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.331696 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.331778 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.332842 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:19.332845 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:19.332847 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:19.332859 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:19.332878 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:19.332886 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:19.332940 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:19.333068 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 03:29:19.333119 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:29:19.334065 30043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.326143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:39239 (local address 127.9.46.195:48291)
0809 03:29:19.326787 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:19.326827 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:19.328045 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:19.328626 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:19.328633 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:19.328752 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:19.329390 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.329412 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.330380 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.330383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.331380 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.331389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.331824 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.331827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.331963 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.332536 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:29:19.332537 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:19.332628 (+ 91us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:29:19.332702 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:19.332952 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:19.332958 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:19.332962 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 03:29:19.333056 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 03:29:19.333277 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":117,"threads_started":1}
I20260809 03:29:19.335158 30041 heartbeater.cc:347] Connected to a master server at 127.9.46.254:39239
I20260809 03:29:19.335397 30041 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:19.335913 30041 heartbeater.cc:511] Master 127.9.46.254:39239 requested a full tablet report, sending...
I20260809 03:29:19.336942 29552 ts_manager.cc:194] Registered new tserver with Master: e0eb3c4549af459b94f5a0d9c5fe4551 (127.9.46.195:42957)
I20260809 03:29:19.337533 29552 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.195:48291
I20260809 03:29:19.342331 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:29:19.347815 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.344255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.9.46.254:39239)
0809 03:29:19.344369 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:19.344373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:19.344525 (+ 152us) server_negotiation.cc:407] Connection header received
0809 03:29:19.344685 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:19.344687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:19.344727 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:19.344793 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:19.345065 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.345562 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.346348 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.346550 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.346623 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.347408 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:19.347411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:19.347413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:19.347426 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:19.347443 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:19.347451 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:19.347550 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:19.347676 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 03:29:19.347707 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 03:29:19.350600 29552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:29:19.353632 29552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:29:19.368252 30055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.363222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:33219 (local address 127.0.0.1:34074)
0809 03:29:19.363924 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:19.363936 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:19.364008 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:19.364445 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:19.364448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:19.364463 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:19.364612 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.364618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.365613 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.365618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.366865 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.366875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.367677 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.367680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.367787 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.367824 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 03:29:19.367967 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 03:29:19.368094 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":9,"thread_start_us":97,"threads_started":1}
I20260809 03:29:19.368257 30060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.363432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34074 (local address 127.9.46.194:33219)
0809 03:29:19.363998 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:19.364008 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:19.364029 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:29:19.364112 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:19.364117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:19.364279 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:19.364400 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:19.364736 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.365490 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.366997 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.367549 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.367675 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.367880 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:19.367987 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 03:29:19.368090 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":132,"threads_started":1}
I20260809 03:29:19.369336 30054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.362365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:39617 (local address 127.0.0.1:47132)
0809 03:29:19.362841 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:19.362895 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:19.363041 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:19.364479 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:19.364482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:19.364504 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:19.364667 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.364674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.366029 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.366035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.367566 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.367577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.368880 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.368884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.368989 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.369011 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 03:29:19.369078 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 03:29:19.369149 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":193,"thread_start_us":96,"threads_started":1}
I20260809 03:29:19.369798 30057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.362506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.9.46.193:39617)
0809 03:29:19.363819 (+ 1313us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:19.363833 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:19.363863 (+ 30us) server_negotiation.cc:407] Connection header received
0809 03:29:19.363947 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:19.363952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:19.364148 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:19.364310 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:19.364807 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.365916 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.367714 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.368515 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.368636 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.368810 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:19.369411 (+ 601us) server_negotiation.cc:299] Negotiation successful
0809 03:29:19.369600 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1196,"thread_start_us":146,"threads_started":1}
I20260809 03:29:19.370221 30059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.362466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:42957 (local address 127.0.0.1:54070)
0809 03:29:19.363613 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:19.363628 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:19.363727 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:19.364274 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:19.364280 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:19.364302 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:19.364465 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.364472 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.365967 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.365973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.367566 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.367577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.369028 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.369032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.369120 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.369143 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 03:29:19.369841 (+ 698us) client_negotiation.cc:240] Negotiation successful
0809 03:29:19.369937 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"mutex_wait_us":236,"thread_start_us":140,"threads_started":1}
I20260809 03:29:19.370725 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.363106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.9.46.195:42957)
0809 03:29:19.363582 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:19.363597 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:19.363762 (+ 165us) server_negotiation.cc:407] Connection header received
0809 03:29:19.363868 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:19.363873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:19.364125 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:19.364286 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:19.364593 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.365801 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.367716 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.368904 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.369009 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.369179 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:19.370421 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0809 03:29:19.370534 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":165,"threads_started":1}
I20260809 03:29:19.371604 29824 tablet_service.cc:1467] Processing CreateTablet for tablet 0a75e6abec954a2aa4011e22f6194549 (DEFAULT_TABLE table=test-table [id=0366de4a3ee04d4ea233f6bb578c11af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:19.372869 29824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a75e6abec954a2aa4011e22f6194549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:19.372572 29684 tablet_service.cc:1467] Processing CreateTablet for tablet 0a75e6abec954a2aa4011e22f6194549 (DEFAULT_TABLE table=test-table [id=0366de4a3ee04d4ea233f6bb578c11af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:19.373608 29684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a75e6abec954a2aa4011e22f6194549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:19.373195 29967 tablet_service.cc:1467] Processing CreateTablet for tablet 0a75e6abec954a2aa4011e22f6194549 (DEFAULT_TABLE table=test-table [id=0366de4a3ee04d4ea233f6bb578c11af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:19.374397 29967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a75e6abec954a2aa4011e22f6194549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:19.380506 30062 tablet_bootstrap.cc:492] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: Bootstrap starting.
I20260809 03:29:19.382776 30063 tablet_bootstrap.cc:492] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: Bootstrap starting.
I20260809 03:29:19.383486 30062 tablet_bootstrap.cc:654] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:19.385293 30063 tablet_bootstrap.cc:654] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:19.385886 30062 log.cc:824] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:19.386847 30064 tablet_bootstrap.cc:492] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44: Bootstrap starting.
I20260809 03:29:19.387056 30063 log.cc:824] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:19.389793 30064 tablet_bootstrap.cc:654] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:19.390003 30063 tablet_bootstrap.cc:492] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: No bootstrap required, opened a new log
I20260809 03:29:19.390312 30063 ts_tablet_manager.cc:1397] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 03:29:19.390865 30064 log.cc:824] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:19.392495 30062 tablet_bootstrap.cc:492] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: No bootstrap required, opened a new log
I20260809 03:29:19.392732 30062 ts_tablet_manager.cc:1397] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.006s
I20260809 03:29:19.394196 30064 tablet_bootstrap.cc:492] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44: No bootstrap required, opened a new log
I20260809 03:29:19.394361 30064 ts_tablet_manager.cc:1397] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260809 03:29:19.397140 30062 raft_consensus.cc:348] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 39617 } } peers { permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 } } peers { permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 } }
I20260809 03:29:19.397426 30062 raft_consensus.cc:374] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:19.397482 30062 raft_consensus.cc:729] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0eb3c4549af459b94f5a0d9c5fe4551, State: Initialized, Role: FOLLOWER
I20260809 03:29:19.397989 30062 consensus_queue.cc:260] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 39617 } } peers { permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 } } peers { permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 } }
I20260809 03:29:19.398218 30063 raft_consensus.cc:348] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 39617 } } peers { permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 } } peers { permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 } }
I20260809 03:29:19.398530 30063 raft_consensus.cc:374] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:19.398587 30063 raft_consensus.cc:729] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f388dfe40f0c4926816d17f8a0c4fa35, State: Initialized, Role: FOLLOWER
I20260809 03:29:19.398480 30064 raft_consensus.cc:348] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 39617 } } peers { permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 } } peers { permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 } }
I20260809 03:29:19.398703 30064 raft_consensus.cc:374] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:19.398751 30064 raft_consensus.cc:729] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f45a70f04244e94aa6cc1c6e827ba44, State: Initialized, Role: FOLLOWER
I20260809 03:29:19.399047 30062 ts_tablet_manager.cc:1428] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260809 03:29:19.399071 30063 consensus_queue.cc:260] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 39617 } } peers { permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 } } peers { permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 } }
I20260809 03:29:19.399137 30064 consensus_queue.cc:260] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 39617 } } peers { permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 } } peers { permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 } }
I20260809 03:29:19.399415 30041 heartbeater.cc:503] Master 127.9.46.254:39239 was elected leader, sending a full tablet report...
I20260809 03:29:19.400008 30064 ts_tablet_manager.cc:1428] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260809 03:29:19.400478 29753 heartbeater.cc:503] Master 127.9.46.254:39239 was elected leader, sending a full tablet report...
I20260809 03:29:19.401755 30063 ts_tablet_manager.cc:1428] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: Time spent starting tablet: real 0.011s user 0.004s sys 0.004s
I20260809 03:29:19.402436 29896 heartbeater.cc:503] Master 127.9.46.254:39239 was elected leader, sending a full tablet report...
W20260809 03:29:19.428864 29755 tablet.cc:2369] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:29:19.575032 30042 tablet.cc:2369] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:29:19.631821 29899 tablet.cc:2369] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:29:19.650068 30069 raft_consensus.cc:482] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:19.650285 30069 raft_consensus.cc:504] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 39617 } } peers { permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 } } peers { permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 } }
I20260809 03:29:19.651423 30069 leader_election.cc:288] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0eb3c4549af459b94f5a0d9c5fe4551 (127.9.46.195:42957), f388dfe40f0c4926816d17f8a0c4fa35 (127.9.46.194:33219)
I20260809 03:29:19.655395 30060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.651807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:50555 (local address 127.9.46.194:33219)
0809 03:29:19.651941 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:19.651948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:19.652169 (+ 221us) server_negotiation.cc:407] Connection header received
0809 03:29:19.652328 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:19.652330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:19.652375 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:19.652455 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:19.652783 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.653263 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.654399 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.654841 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.654974 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.655031 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:19.655216 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 03:29:19.655287 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:29:19.655418 30071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.651697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:33219 (local address 127.9.46.193:50555)
0809 03:29:19.652077 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:19.652088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:19.652212 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:19.652478 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:19.652480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:19.652496 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:19.652668 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.652674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.653377 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.653382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.654274 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.654284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.654974 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.654980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.655074 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.655107 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:29:19.655189 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 03:29:19.655262 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":26,"thread_start_us":84,"threads_started":1}
I20260809 03:29:19.655552 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.651776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:52557 (local address 127.9.46.195:42957)
0809 03:29:19.651906 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:19.651912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:19.652109 (+ 197us) server_negotiation.cc:407] Connection header received
0809 03:29:19.652281 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:19.652289 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:19.652328 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:19.652411 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:19.652733 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.653254 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.654401 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.654924 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.655032 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.655078 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:19.655352 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 03:29:19.655425 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:29:19.655931 30073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.651674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:42957 (local address 127.9.46.193:52557)
0809 03:29:19.652018 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:19.652030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:19.652172 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:19.652425 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:19.652428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:19.652444 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:19.652630 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.652636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.653367 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.653371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.654285 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:19.654294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.655111 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.655114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:19.655194 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.655222 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 03:29:19.655701 (+ 479us) client_negotiation.cc:240] Negotiation successful
0809 03:29:19.655776 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":67,"thread_start_us":106,"threads_started":1}
I20260809 03:29:19.656200 29846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a75e6abec954a2aa4011e22f6194549" candidate_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" is_pre_election: true
I20260809 03:29:19.656493 29846 raft_consensus.cc:2454] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f45a70f04244e94aa6cc1c6e827ba44 in term 0.
I20260809 03:29:19.656518 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a75e6abec954a2aa4011e22f6194549" candidate_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" is_pre_election: true
I20260809 03:29:19.656847 29991 raft_consensus.cc:2454] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f45a70f04244e94aa6cc1c6e827ba44 in term 0.
I20260809 03:29:19.656985 29637 leader_election.cc:302] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f45a70f04244e94aa6cc1c6e827ba44, f388dfe40f0c4926816d17f8a0c4fa35; no voters:
I20260809 03:29:19.657274 30069 raft_consensus.cc:2781] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:29:19.657368 30069 raft_consensus.cc:482] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:29:19.657428 30069 raft_consensus.cc:3037] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:19.658417 30069 raft_consensus.cc:504] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 39617 } } peers { permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 } } peers { permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 } }
I20260809 03:29:19.658754 30069 leader_election.cc:288] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [CANDIDATE]: Term 1 election: Requested vote from peers e0eb3c4549af459b94f5a0d9c5fe4551 (127.9.46.195:42957), f388dfe40f0c4926816d17f8a0c4fa35 (127.9.46.194:33219)
I20260809 03:29:19.659147 29846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a75e6abec954a2aa4011e22f6194549" candidate_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f388dfe40f0c4926816d17f8a0c4fa35"
I20260809 03:29:19.659291 29846 raft_consensus.cc:3037] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:19.660359 29846 raft_consensus.cc:2454] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f45a70f04244e94aa6cc1c6e827ba44 in term 1.
I20260809 03:29:19.660701 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a75e6abec954a2aa4011e22f6194549" candidate_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551"
I20260809 03:29:19.660743 29637 leader_election.cc:302] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f45a70f04244e94aa6cc1c6e827ba44, f388dfe40f0c4926816d17f8a0c4fa35; no voters:
I20260809 03:29:19.660841 29991 raft_consensus.cc:3037] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:19.660921 30069 raft_consensus.cc:2781] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:19.661164 30069 raft_consensus.cc:686] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [term 1 LEADER]: Becoming Leader. State: Replica: 1f45a70f04244e94aa6cc1c6e827ba44, State: Running, Role: LEADER
I20260809 03:29:19.661533 30069 consensus_queue.cc:237] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 39617 } } peers { permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 } } peers { permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 } }
I20260809 03:29:19.662143 29991 raft_consensus.cc:2454] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f45a70f04244e94aa6cc1c6e827ba44 in term 1.
I20260809 03:29:19.664817 29552 catalog_manager.cc:5495] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f45a70f04244e94aa6cc1c6e827ba44 (127.9.46.193). New cstate: current_term: 1 leader_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f45a70f04244e94aa6cc1c6e827ba44" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 39617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 } health_report { overall_health: UNKNOWN } } }
I20260809 03:29:19.735419 30057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.731798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.9.46.193:39617)
0809 03:29:19.731946 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:19.731953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:19.731969 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:29:19.732055 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:19.732057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:19.732115 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:19.732253 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:19.732512 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.733032 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.733903 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:19.734122 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:19.734241 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:19.734569 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:19.734978 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:19.735149 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:19.735235 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 03:29:19.735275 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:29:19.741211 29991 raft_consensus.cc:1261] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551 [term 1 FOLLOWER]: Refusing update from remote peer 1f45a70f04244e94aa6cc1c6e827ba44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:19.741215 29846 raft_consensus.cc:1261] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35 [term 1 FOLLOWER]: Refusing update from remote peer 1f45a70f04244e94aa6cc1c6e827ba44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:19.741936 30074 consensus_queue.cc:1035] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0eb3c4549af459b94f5a0d9c5fe4551" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:19.742129 30069 consensus_queue.cc:1035] T 0a75e6abec954a2aa4011e22f6194549 P 1f45a70f04244e94aa6cc1c6e827ba44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f388dfe40f0c4926816d17f8a0c4fa35" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 33219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:19.748924 30077 mvcc.cc:205] Tried to move back new op lower bound from 7316464270292484096 to 7316464269976702976. Current Snapshot: MvccSnapshot[applied={T|T < 7316464270292484096}]
I20260809 03:29:19.757632 29552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:19.757825 29552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:19.760116 29552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 998d5080f51a4cca85502c56ce6532a5: Sending DeleteTablet for 3 replicas of tablet 0a75e6abec954a2aa4011e22f6194549
I20260809 03:29:19.760953 29684 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a75e6abec954a2aa4011e22f6194549 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:19 UTC) from {username='slave'} at 127.0.0.1:47132
I20260809 03:29:19.760972 29967 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a75e6abec954a2aa4011e22f6194549 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:19 UTC) from {username='slave'} at 127.0.0.1:54070
I20260809 03:29:19.760972 29824 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a75e6abec954a2aa4011e22f6194549 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:19 UTC) from {username='slave'} at 127.0.0.1:34074
I20260809 03:29:19.761622 30088 tablet_replica.cc:330] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: stopping tablet replica
I20260809 03:29:19.762061 30087 tablet_replica.cc:330] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: stopping tablet replica
I20260809 03:29:19.762147 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 29617
I20260809 03:29:19.764263 30088 raft_consensus.cc:2229] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:29:19.764271 30087 raft_consensus.cc:2229] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:29:19.764624 30088 raft_consensus.cc:2258] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:19.764634 30087 raft_consensus.cc:2258] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:19.766156 30087 ts_tablet_manager.cc:1905] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:29:19.766446 30088 ts_tablet_manager.cc:1905] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:29:19.768589 30087 ts_tablet_manager.cc:1918] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:29:19.768709 30087 log.cc:1196] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/0a75e6abec954a2aa4011e22f6194549
I20260809 03:29:19.769017 30087 ts_tablet_manager.cc:1939] T 0a75e6abec954a2aa4011e22f6194549 P f388dfe40f0c4926816d17f8a0c4fa35: Deleting consensus metadata
I20260809 03:29:19.769918 30088 ts_tablet_manager.cc:1918] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:29:19.770017 30088 log.cc:1196] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/0a75e6abec954a2aa4011e22f6194549
I20260809 03:29:19.770126 29539 catalog_manager.cc:4841] TS f388dfe40f0c4926816d17f8a0c4fa35 (127.9.46.194:33219): tablet 0a75e6abec954a2aa4011e22f6194549 (table test-table [id=0366de4a3ee04d4ea233f6bb578c11af]) successfully deleted
I20260809 03:29:19.770347 30088 ts_tablet_manager.cc:1939] T 0a75e6abec954a2aa4011e22f6194549 P e0eb3c4549af459b94f5a0d9c5fe4551: Deleting consensus metadata
I20260809 03:29:19.771520 29536 catalog_manager.cc:4841] TS e0eb3c4549af459b94f5a0d9c5fe4551 (127.9.46.195:42957): tablet 0a75e6abec954a2aa4011e22f6194549 (table test-table [id=0366de4a3ee04d4ea233f6bb578c11af]) successfully deleted
W20260809 03:29:19.771803 29537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:39617 (error 108)
I20260809 03:29:19.772672 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 29758
I20260809 03:29:19.772872 30059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:19.772525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:39617 (local address 127.0.0.1:47156)
0809 03:29:19.772671 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:19.772773 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:39617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 03:29:19.774430 29537 catalog_manager.cc:4584] TS 1f45a70f04244e94aa6cc1c6e827ba44 (127.9.46.193:39617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a75e6abec954a2aa4011e22f6194549: Network error: Client connection negotiation failed: client connection to 127.9.46.193:39617: connect: Connection refused (error 111)
I20260809 03:29:19.780594 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 29901
I20260809 03:29:19.788604 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 29520
2026-08-09T03:29:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1410 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:19 dist-test-slave-65mx krb5kdc[30092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:19 dist-test-slave-65mx krb5kdc[30092](info): set up 2 sockets
Aug 09 03:29:19 dist-test-slave-65mx krb5kdc[30092](info): commencing operation
krb5kdc: starting...
W20260809 03:29:20.872233 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:20 dist-test-slave-65mx krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:21.003129 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:44253
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:42109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:44253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:21.113153 30106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:21.113448 30106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:21.113518 30106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:21.117516 30106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:21.117587 30106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:21.117624 30106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:21.117673 30106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:21.117709 30106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:21.122933 30106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:42109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:44253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:44253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:21.123961 30106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:21.125000 30106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:21.133283 30114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:21.133293 30111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:21.133455 30106 server_base.cc:1034] running on GCE node
W20260809 03:29:21.133258 30112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:21.134192 30106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:21.135299 30106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:21.136461 30106 hybrid_clock.cc:648] HybridClock initialized: now 1786246161136450 us; error 28 us; skew 500 ppm
Aug 09 03:29:21 dist-test-slave-65mx krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246161, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:21.139207 30106 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:21.140082 30106 webserver.cc:460] Webserver started at http://127.9.46.254:45079/ using document root <none> and password file <none>
I20260809 03:29:21.140686 30106 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:21.140775 30106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:21.141000 30106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:21.142970 30106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "ade63e80a1f847998a8ea5a678636885"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "5fa7d383ce0d41faf06d6ba13571cff7"
server_key_iv: "431b008a78e4e7bec87a2cfa73efb47b"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.143532 30106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "ade63e80a1f847998a8ea5a678636885"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "5fa7d383ce0d41faf06d6ba13571cff7"
server_key_iv: "431b008a78e4e7bec87a2cfa73efb47b"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.147210 30106 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 03:29:21.149657 30122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:21.150844 30106 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:29:21.150979 30106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "ade63e80a1f847998a8ea5a678636885"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "5fa7d383ce0d41faf06d6ba13571cff7"
server_key_iv: "431b008a78e4e7bec87a2cfa73efb47b"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.151093 30106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:21.187453 30106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:21.190796 30106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:21.191023 30106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:21.198446 30106 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:44253
I20260809 03:29:21.198470 30177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:44253 every 8 connection(s)
I20260809 03:29:21.199429 30106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:21.200217 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 30106
I20260809 03:29:21.200366 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:21.200732 9403 external_mini_cluster.cc:1428] Setting key 758df9a9e4276bd0da47418b1f5be5dd
I20260809 03:29:21.202703 30178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:21.209281 30178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885: Bootstrap starting.
Aug 09 03:29:21 dist-test-slave-65mx krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:21.212059 30178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:21.213009 30178 log.cc:824] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:21.215378 30178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885: No bootstrap required, opened a new log
I20260809 03:29:21.215652 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.202243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50770 (local address 127.9.46.254:44253)
0809 03:29:21.202641 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:21.202658 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:21.202699 (+ 41us) server_negotiation.cc:407] Connection header received
0809 03:29:21.203355 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:21.203371 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:21.203684 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:21.203954 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:21.204889 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.205882 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.207039 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.207487 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.207607 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.210460 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:21.210467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:21.210481 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:21.210522 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:21.212646 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:21.213071 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:21.213077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:21.213084 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:21.213221 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:21.213504 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:21.213507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:21.213509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:21.213984 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:21.214192 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:21.214425 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 03:29:21.214718 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":259,"thread_start_us":137,"threads_started":1}
I20260809 03:29:21.219355 30178 raft_consensus.cc:348] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade63e80a1f847998a8ea5a678636885" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44253 } }
I20260809 03:29:21.219631 30178 raft_consensus.cc:374] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:21.219697 30178 raft_consensus.cc:729] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade63e80a1f847998a8ea5a678636885, State: Initialized, Role: FOLLOWER
I20260809 03:29:21.220129 30178 consensus_queue.cc:260] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade63e80a1f847998a8ea5a678636885" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44253 } }
I20260809 03:29:21.220271 30178 raft_consensus.cc:388] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:21.220340 30178 raft_consensus.cc:482] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:21.220435 30178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:21.221400 30178 raft_consensus.cc:504] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade63e80a1f847998a8ea5a678636885" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44253 } }
I20260809 03:29:21.221748 30178 leader_election.cc:302] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ade63e80a1f847998a8ea5a678636885; no voters:
I20260809 03:29:21.222100 30178 leader_election.cc:288] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:21.222252 30183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:21.222482 30183 raft_consensus.cc:686] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [term 1 LEADER]: Becoming Leader. State: Replica: ade63e80a1f847998a8ea5a678636885, State: Running, Role: LEADER
I20260809 03:29:21.222857 30183 consensus_queue.cc:237] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade63e80a1f847998a8ea5a678636885" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44253 } }
I20260809 03:29:21.223362 30178 sys_catalog.cc:564] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:21.224586 30185 sys_catalog.cc:455] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ade63e80a1f847998a8ea5a678636885. Latest consensus state: current_term: 1 leader_uuid: "ade63e80a1f847998a8ea5a678636885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade63e80a1f847998a8ea5a678636885" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44253 } } }
I20260809 03:29:21.224624 30184 sys_catalog.cc:455] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ade63e80a1f847998a8ea5a678636885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade63e80a1f847998a8ea5a678636885" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44253 } } }
I20260809 03:29:21.224721 30185 sys_catalog.cc:458] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:21.224725 30184 sys_catalog.cc:458] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:21.225116 30192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:21.228346 30192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:21.234090 30192 catalog_manager.cc:1347] Generated new cluster ID: 5bfe97d915a04f4385eeb2f8d37381d7
I20260809 03:29:21.234182 30192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:21.240711 30192 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:21.241585 30192 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:21.248610 30192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885: Generated new TSK 0
I20260809 03:29:21.249266 30192 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.46.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
Entry for principal kudu/127.9.46.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab.
I20260809 03:29:21.281669 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44253
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:42109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:21.393792 30205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:21.394062 30205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:21.394106 30205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:21.398243 30205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:21.398301 30205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:21.398389 30205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:29:21.403632 30205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:42109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.193.keytab
--principal=kudu/127.9.46.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:21.404776 30205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:21.405771 30205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:21.416980 30210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:21.417053 30213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:21.417186 30205 server_base.cc:1034] running on GCE node
W20260809 03:29:21.417354 30211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:21.417786 30205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:21.418433 30205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:21.419620 30205 hybrid_clock.cc:648] HybridClock initialized: now 1786246161419607 us; error 26 us; skew 500 ppm
Aug 09 03:29:21 dist-test-slave-65mx krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246161, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:21.422574 30205 init.cc:377] Logged in from keytab as kudu/127.9.46.193@KRBTEST.COM (short username kudu)
I20260809 03:29:21.423457 30205 webserver.cc:460] Webserver started at http://127.9.46.193:44915/ using document root <none> and password file <none>
I20260809 03:29:21.424062 30205 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:21.424125 30205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:21.424301 30205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:21.426435 30205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "094d1c6a392248899e99883ab8a9655b"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "e03c3efd0d495dadb991c5b132974571"
server_key_iv: "0da7f1a242b957204a894e4c7956fc92"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.426981 30205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "094d1c6a392248899e99883ab8a9655b"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "e03c3efd0d495dadb991c5b132974571"
server_key_iv: "0da7f1a242b957204a894e4c7956fc92"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.430750 30205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 03:29:21.433177 30220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:21.434412 30205 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:21.434535 30205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "094d1c6a392248899e99883ab8a9655b"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "e03c3efd0d495dadb991c5b132974571"
server_key_iv: "0da7f1a242b957204a894e4c7956fc92"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.434645 30205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:21.463263 30205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:21.466398 30205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:21.466605 30205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:21.467235 30205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:21.468115 30205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:21.468211 30205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:21.468302 30205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:21.468348 30205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:21.477448 30205 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:42721
I20260809 03:29:21.477469 30341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:42721 every 8 connection(s)
I20260809 03:29:21.478360 30205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:29:21.478899 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 30205
I20260809 03:29:21.479048 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:29:21.479352 9403 external_mini_cluster.cc:1428] Setting key ca1614d72763778793bbef9b18bd6f5b
Aug 09 03:29:21 dist-test-slave-65mx krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246161, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.193@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.46.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.194@KRBTEST.COM" created.
I20260809 03:29:21.493844 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.480254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:54819 (local address 127.9.46.254:44253)
0809 03:29:21.480395 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:21.480400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:21.481463 (+ 1063us) server_negotiation.cc:407] Connection header received
0809 03:29:21.483398 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:21.483405 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:21.483470 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:21.483559 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:21.484956 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.485852 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.487090 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.487552 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.487702 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.490359 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:21.490362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:21.490364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:21.490389 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:21.491765 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:21.492252 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:21.492255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:21.492256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:21.492330 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:21.492746 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:21.492749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:21.492750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:21.492928 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:21.493029 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:21.493583 (+ 554us) server_negotiation.cc:299] Negotiation successful
0809 03:29:21.493718 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 03:29:21.494974 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.480739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44253 (local address 127.9.46.193:54819)
0809 03:29:21.481289 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:21.481348 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:21.483127 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:21.483701 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:21.483707 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:21.484054 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:21.484533 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.484608 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.485996 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.485999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.486954 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.486964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.487721 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.487724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.487863 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.488484 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:21.488501 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:21.490183 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:21.491873 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:21.491878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:21.491890 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:21.492150 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:21.492423 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:21.492425 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:21.492427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:21.492656 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:21.493012 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:21.493017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:21.493317 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 03:29:21.493488 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 03:29:21.493851 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":151,"threads_started":1}
I20260809 03:29:21.496313 30342 heartbeater.cc:347] Connected to a master server at 127.9.46.254:44253
I20260809 03:29:21.496609 30342 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:21.497136 30342 heartbeater.cc:511] Master 127.9.46.254:44253 requested a full tablet report, sending...
I20260809 03:29:21.498858 30140 ts_manager.cc:194] Registered new tserver with Master: 094d1c6a392248899e99883ab8a9655b (127.9.46.193:42721)
I20260809 03:29:21.500649 30140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.193@KRBTEST.COM'} at 127.9.46.193:54819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
Entry for principal kudu/127.9.46.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab.
I20260809 03:29:21.512434 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44253
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:42109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:21.624059 30348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:21.624382 30348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:21.624432 30348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:21.628526 30348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:21.628608 30348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:21.628669 30348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:29:21.633940 30348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:42109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.194.keytab
--principal=kudu/127.9.46.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:21.634984 30348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:21.635968 30348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 03:29:21.645812 30348 server_base.cc:1034] running on GCE node
W20260809 03:29:21.645740 30354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:21.645850 30353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:21.645744 30356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:21.646564 30348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:21.647204 30348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:21.648386 30348 hybrid_clock.cc:648] HybridClock initialized: now 1786246161648360 us; error 42 us; skew 500 ppm
Aug 09 03:29:21 dist-test-slave-65mx krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246161, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:21.651266 30348 init.cc:377] Logged in from keytab as kudu/127.9.46.194@KRBTEST.COM (short username kudu)
I20260809 03:29:21.652128 30348 webserver.cc:460] Webserver started at http://127.9.46.194:46001/ using document root <none> and password file <none>
I20260809 03:29:21.652815 30348 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:21.652912 30348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:21.653110 30348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:21.655243 30348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "dd0d4d2269f34f4e9ed1d188b2e00744"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "e5814981d1b7d0dad8f35582c6f3b921"
server_key_iv: "909604e15d378e0e27fa470adbf89b1c"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.655788 30348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "dd0d4d2269f34f4e9ed1d188b2e00744"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "e5814981d1b7d0dad8f35582c6f3b921"
server_key_iv: "909604e15d378e0e27fa470adbf89b1c"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.659487 30348 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 03:29:21.661834 30363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:21.663000 30348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:29:21.663137 30348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "dd0d4d2269f34f4e9ed1d188b2e00744"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "e5814981d1b7d0dad8f35582c6f3b921"
server_key_iv: "909604e15d378e0e27fa470adbf89b1c"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.663249 30348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:21.673847 30348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:21.676657 30348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:21.676841 30348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:21.677408 30348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:21.678195 30348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:21.678262 30348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:21.678329 30348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:21.678376 30348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:21.686697 30348 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:37941
I20260809 03:29:21.686762 30488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:37941 every 8 connection(s)
I20260809 03:29:21.687585 30348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:29:21.689808 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 30348
I20260809 03:29:21.689958 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:29:21.690321 9403 external_mini_cluster.cc:1428] Setting key cfab63abfb9dfaf0f2d97fa8ecd9930b
Aug 09 03:29:21 dist-test-slave-65mx krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246161, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.194@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.46.195@KRBTEST.COM; defaulting to no policy
I20260809 03:29:21.702083 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.689564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:34917 (local address 127.9.46.254:44253)
0809 03:29:21.689729 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:21.689734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:21.690455 (+ 721us) server_negotiation.cc:407] Connection header received
0809 03:29:21.692123 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:21.692126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:21.692173 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:21.692242 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:21.693853 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.694674 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.695810 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.696051 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.696184 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.698585 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:21.698593 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:21.698595 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:21.698618 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:21.700034 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:21.700526 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:21.700531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:21.700533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:21.700653 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:21.700978 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:21.700983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:21.700985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:21.701213 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:21.701325 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:21.701795 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 03:29:21.701915 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 03:29:21.702875 30491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.689733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44253 (local address 127.9.46.194:34917)
0809 03:29:21.690290 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:21.690347 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:21.691902 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:21.692417 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:21.692428 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:21.692981 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:21.693697 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.693718 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.694819 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.694822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.695693 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.695702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.696209 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.696214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.696396 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.697018 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:21.697034 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:21.698425 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:21.700152 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:21.700158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:21.700172 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:21.700423 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:21.700743 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:21.700747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:21.700749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:21.700884 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:21.701322 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:21.701328 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:21.701562 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 03:29:21.701747 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 03:29:21.702103 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":146,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.195@KRBTEST.COM" created.
I20260809 03:29:21.704103 30489 heartbeater.cc:347] Connected to a master server at 127.9.46.254:44253
I20260809 03:29:21.704341 30489 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:21.704847 30489 heartbeater.cc:511] Master 127.9.46.254:44253 requested a full tablet report, sending...
I20260809 03:29:21.705850 30140 ts_manager.cc:194] Registered new tserver with Master: dd0d4d2269f34f4e9ed1d188b2e00744 (127.9.46.194:37941)
I20260809 03:29:21.706398 30140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.194@KRBTEST.COM'} at 127.9.46.194:34917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
Entry for principal kudu/127.9.46.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab.
I20260809 03:29:21.720003 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44253
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:42109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:21.833364 30494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:21.833628 30494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:21.833670 30494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:21.837816 30494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:21.837870 30494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:21.837927 30494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:29:21.843478 30494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:42109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.195.keytab
--principal=kudu/127.9.46.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:44253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:21.844422 30494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:21.845407 30494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:21.854852 30501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:21.854852 30500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:21.854954 30504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:21.855330 30494 server_base.cc:1034] running on GCE node
I20260809 03:29:21.855721 30494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:21.856376 30494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:21.857579 30494 hybrid_clock.cc:648] HybridClock initialized: now 1786246161857579 us; error 41 us; skew 500 ppm
Aug 09 03:29:21 dist-test-slave-65mx krb5kdc[30092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246161, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:21.860342 30494 init.cc:377] Logged in from keytab as kudu/127.9.46.195@KRBTEST.COM (short username kudu)
I20260809 03:29:21.861277 30494 webserver.cc:460] Webserver started at http://127.9.46.195:45425/ using document root <none> and password file <none>
I20260809 03:29:21.861824 30494 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:21.861912 30494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:21.862136 30494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:21.864125 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "f96c45e6cd1b4074ac6b19cb8d652025"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "a1d18c1ae56af9c53b596b868861eb11"
server_key_iv: "fbdb62102b541df5fb859a53064bb0cd"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.864739 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "f96c45e6cd1b4074ac6b19cb8d652025"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "a1d18c1ae56af9c53b596b868861eb11"
server_key_iv: "fbdb62102b541df5fb859a53064bb0cd"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.868465 30494 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260809 03:29:21.870865 30512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:21.872013 30494 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:21.872156 30494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "f96c45e6cd1b4074ac6b19cb8d652025"
format_stamp: "Formatted at 2026-08-09 03:29:21 on dist-test-slave-65mx"
server_key: "a1d18c1ae56af9c53b596b868861eb11"
server_key_iv: "fbdb62102b541df5fb859a53064bb0cd"
server_key_version: "encryptionkey@0"
I20260809 03:29:21.872287 30494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:21.887306 30494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:21.890115 30494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:21.890323 30494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:21.890925 30494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:21.891750 30494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:21.891829 30494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:21.891903 30494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:21.891952 30494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:21.900586 30494 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:35417
I20260809 03:29:21.900620 30635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:35417 every 8 connection(s)
I20260809 03:29:21.901500 30494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:29:21.907068 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 30494
I20260809 03:29:21.907224 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:29:21.907548 9403 external_mini_cluster.cc:1428] Setting key 8bfba630cf40d3ef117341aca24bc13b
Aug 09 03:29:21 dist-test-slave-65mx krb5kdc[30092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246161, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.195@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:21.915571 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.903291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:51939 (local address 127.9.46.254:44253)
0809 03:29:21.903438 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:21.903442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:21.904342 (+ 900us) server_negotiation.cc:407] Connection header received
0809 03:29:21.905798 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:21.905802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:21.905865 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:21.905953 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:21.907225 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.908043 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.909510 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.909764 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.909884 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.912299 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:21.912303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:21.912305 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:21.912336 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:21.913629 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:21.914112 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:21.914115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:21.914117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:21.914205 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:21.914531 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:21.914535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:21.914536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:21.914724 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:21.914828 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:21.915291 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 03:29:21.915406 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:29:21.916303 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.903636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44253 (local address 127.9.46.195:51939)
0809 03:29:21.904169 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:21.904214 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:21.905574 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:21.906130 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:21.906137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:21.906527 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:21.907055 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.907070 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.908168 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.908171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.909373 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.909383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.909923 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.909927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.910120 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.910685 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:21.910709 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:21.912129 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:21.913749 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:21.913754 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:21.913767 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:21.914011 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:21.914301 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:21.914304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:21.914306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:21.914439 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:21.914826 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:21.914833 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:21.915065 (+ 232us) client_negotiation.cc:769] Sending connection context
0809 03:29:21.915249 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 03:29:21.915539 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":117,"threads_started":1}
I20260809 03:29:21.917354 30637 heartbeater.cc:347] Connected to a master server at 127.9.46.254:44253
I20260809 03:29:21.917544 30637 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:21.917946 30637 heartbeater.cc:511] Master 127.9.46.254:44253 requested a full tablet report, sending...
I20260809 03:29:21.918855 30140 ts_manager.cc:194] Registered new tserver with Master: f96c45e6cd1b4074ac6b19cb8d652025 (127.9.46.195:35417)
I20260809 03:29:21.919360 30140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.46.195@KRBTEST.COM'} at 127.9.46.195:51939
I20260809 03:29:21.921648 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:29:21.931207 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.923843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50774 (local address 127.9.46.254:44253)
0809 03:29:21.923976 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:21.923980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:21.924263 (+ 283us) server_negotiation.cc:407] Connection header received
0809 03:29:21.924364 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:21.924366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:21.924401 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:21.924462 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:21.925231 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.925706 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.926594 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.926802 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.926875 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.928210 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:21.928213 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:21.928215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:21.928235 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:21.929652 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:21.930073 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:21.930078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:21.930081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:21.930166 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:21.930395 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:21.930397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:21.930398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:21.930697 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:21.930798 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:21.930959 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 03:29:21.931041 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:29:21.933876 30140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:29:21.937479 30140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:29:21.951609 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.946454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36868 (local address 127.9.46.194:37941)
0809 03:29:21.946776 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:21.946788 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:21.947306 (+ 518us) server_negotiation.cc:407] Connection header received
0809 03:29:21.947392 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:21.947396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:21.947527 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:21.947669 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:21.947991 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.948853 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.950286 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.950889 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.951011 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.951158 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:21.951340 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 03:29:21.951442 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":97,"threads_started":1}
I20260809 03:29:21.951761 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.945598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:42721 (local address 127.0.0.1:39386)
0809 03:29:21.946159 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:21.946172 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:21.946259 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:21.947688 (+ 1429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:21.947691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:21.947709 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:21.947847 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.947853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.949146 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.949149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.950305 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.950313 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.951367 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.951370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.951455 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.951475 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 03:29:21.951525 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 03:29:21.951581 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":64,"thread_start_us":85,"threads_started":1}
I20260809 03:29:21.951630 30651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.945475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:35417 (local address 127.0.0.1:35754)
0809 03:29:21.945976 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:21.946013 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:21.946147 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:21.946934 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:21.946941 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:21.946965 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:21.947133 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.947143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.948583 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.948586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.950039 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.950050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.951124 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.951128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.951225 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.951250 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 03:29:21.951301 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 03:29:21.951449 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":29,"thread_start_us":73,"threads_started":1}
I20260809 03:29:21.952086 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.945601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.9.46.195:35417)
0809 03:29:21.946434 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:21.946446 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:21.946468 (+ 22us) server_negotiation.cc:407] Connection header received
0809 03:29:21.946559 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:21.946566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:21.946769 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:21.946927 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:21.947277 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.948402 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.950162 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.950976 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.951691 (+ 715us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.951813 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:21.951863 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 03:29:21.951945 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":140,"threads_started":1}
I20260809 03:29:21.952086 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.946591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39386 (local address 127.9.46.193:42721)
0809 03:29:21.947242 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:21.947258 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:21.947279 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:29:21.947351 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:21.947355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:21.947521 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:21.947981 (+ 460us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:21.948137 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.949013 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.950422 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.951233 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.951675 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.951813 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:21.951863 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 03:29:21.951944 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":119,"threads_started":1}
I20260809 03:29:21.954659 30271 tablet_service.cc:1467] Processing CreateTablet for tablet 4ebe255e5ab04e46a22d98130f4bb59e (DEFAULT_TABLE table=test-table [id=e1508cdc614745fc8c3abee0064972da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:21.954660 30565 tablet_service.cc:1467] Processing CreateTablet for tablet 4ebe255e5ab04e46a22d98130f4bb59e (DEFAULT_TABLE table=test-table [id=e1508cdc614745fc8c3abee0064972da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:21.955621 30271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ebe255e5ab04e46a22d98130f4bb59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:21.955624 30565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ebe255e5ab04e46a22d98130f4bb59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:21.951601 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:21.946176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:37941 (local address 127.0.0.1:36868)
0809 03:29:21.947072 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:21.947091 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:21.947208 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:21.947649 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:21.947652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:21.947669 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:21.947862 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.947869 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.948992 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.948995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.950164 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:21.950175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:21.951012 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:21.951016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:21.951143 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:21.951185 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 03:29:21.951309 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 03:29:21.951454 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":142,"thread_start_us":95,"threads_started":1}
I20260809 03:29:21.961580 30656 tablet_bootstrap.cc:492] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b: Bootstrap starting.
I20260809 03:29:21.962419 30657 tablet_bootstrap.cc:492] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: Bootstrap starting.
I20260809 03:29:21.963667 30656 tablet_bootstrap.cc:654] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:21.963577 30415 tablet_service.cc:1467] Processing CreateTablet for tablet 4ebe255e5ab04e46a22d98130f4bb59e (DEFAULT_TABLE table=test-table [id=e1508cdc614745fc8c3abee0064972da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:21.964452 30657 tablet_bootstrap.cc:654] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:21.964689 30415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ebe255e5ab04e46a22d98130f4bb59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:21.964864 30656 log.cc:824] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:21.965715 30657 log.cc:824] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:21.966747 30656 tablet_bootstrap.cc:492] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b: No bootstrap required, opened a new log
I20260809 03:29:21.966934 30656 ts_tablet_manager.cc:1397] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 03:29:21.967722 30657 tablet_bootstrap.cc:492] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: No bootstrap required, opened a new log
I20260809 03:29:21.967890 30657 ts_tablet_manager.cc:1397] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:29:21.970831 30656 raft_consensus.cc:348] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35417 } } peers { permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 } } peers { permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 } }
I20260809 03:29:21.971194 30656 raft_consensus.cc:374] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:21.971212 30660 tablet_bootstrap.cc:492] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744: Bootstrap starting.
I20260809 03:29:21.971304 30656 raft_consensus.cc:729] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094d1c6a392248899e99883ab8a9655b, State: Initialized, Role: FOLLOWER
I20260809 03:29:21.971664 30657 raft_consensus.cc:348] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35417 } } peers { permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 } } peers { permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 } }
I20260809 03:29:21.971845 30656 consensus_queue.cc:260] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35417 } } peers { permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 } } peers { permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 } }
I20260809 03:29:21.971922 30657 raft_consensus.cc:374] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:21.971974 30657 raft_consensus.cc:729] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f96c45e6cd1b4074ac6b19cb8d652025, State: Initialized, Role: FOLLOWER
I20260809 03:29:21.972397 30657 consensus_queue.cc:260] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35417 } } peers { permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 } } peers { permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 } }
I20260809 03:29:21.972623 30342 heartbeater.cc:503] Master 127.9.46.254:44253 was elected leader, sending a full tablet report...
I20260809 03:29:21.972925 30656 ts_tablet_manager.cc:1428] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260809 03:29:21.973269 30657 ts_tablet_manager.cc:1428] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:29:21.973548 30637 heartbeater.cc:503] Master 127.9.46.254:44253 was elected leader, sending a full tablet report...
I20260809 03:29:21.973857 30660 tablet_bootstrap.cc:654] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:21.975008 30660 log.cc:824] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:21.977053 30660 tablet_bootstrap.cc:492] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744: No bootstrap required, opened a new log
I20260809 03:29:21.977272 30660 ts_tablet_manager.cc:1397] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
W20260809 03:29:21.979450 30344 tablet.cc:2369] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:29:21.980072 30660 raft_consensus.cc:348] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35417 } } peers { permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 } } peers { permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 } }
I20260809 03:29:21.980293 30660 raft_consensus.cc:374] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:21.980360 30660 raft_consensus.cc:729] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd0d4d2269f34f4e9ed1d188b2e00744, State: Initialized, Role: FOLLOWER
I20260809 03:29:21.980758 30660 consensus_queue.cc:260] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35417 } } peers { permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 } } peers { permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 } }
I20260809 03:29:21.981345 30489 heartbeater.cc:503] Master 127.9.46.254:44253 was elected leader, sending a full tablet report...
I20260809 03:29:21.981614 30660 ts_tablet_manager.cc:1428] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
W20260809 03:29:22.152514 30639 tablet.cc:2369] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 03:29:22.188760 30490 tablet.cc:2369] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:29:22.296131 30662 raft_consensus.cc:482] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:22.296346 30662 raft_consensus.cc:504] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35417 } } peers { permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 } } peers { permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 } }
I20260809 03:29:22.297624 30662 leader_election.cc:288] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 094d1c6a392248899e99883ab8a9655b (127.9.46.193:42721), dd0d4d2269f34f4e9ed1d188b2e00744 (127.9.46.194:37941)
I20260809 03:29:22.301519 30640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:22.297891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:42721 (local address 127.9.46.195:48979)
0809 03:29:22.298030 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:22.298047 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:22.298149 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:22.298473 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:22.298476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:22.298496 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:22.298630 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:22.298636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.299434 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.299439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:22.300297 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:22.300306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.301058 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.301062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:22.301188 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:22.301206 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 03:29:22.301300 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 03:29:22.301375 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":1}
I20260809 03:29:22.301519 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:22.298012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:48979 (local address 127.9.46.193:42721)
0809 03:29:22.298170 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:22.298175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:22.298200 (+ 25us) server_negotiation.cc:407] Connection header received
0809 03:29:22.298271 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:22.298274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:22.298332 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:22.298436 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:22.298758 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.299314 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.300426 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.300932 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.301068 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:22.301126 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:22.301319 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 03:29:22.301394 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:29:22.301721 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:22.298343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:42187 (local address 127.9.46.194:37941)
0809 03:29:22.298501 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:22.298507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:22.298748 (+ 241us) server_negotiation.cc:407] Connection header received
0809 03:29:22.298868 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:22.298870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:22.298917 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:22.298995 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:22.299258 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.299734 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.300827 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.301197 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.301318 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:22.301366 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:22.301544 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 03:29:22.301610 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 03:29:22.301730 30665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:22.298201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:37941 (local address 127.9.46.195:42187)
0809 03:29:22.298658 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:22.298669 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:22.298780 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:22.299012 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:22.299015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:22.299030 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:22.299148 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:22.299153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.299837 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.299842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:22.300710 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:22.300719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.301309 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.301313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:22.301417 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:22.301438 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 03:29:22.301527 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 03:29:22.301591 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":151,"threads_started":1}
I20260809 03:29:22.302282 30293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebe255e5ab04e46a22d98130f4bb59e" candidate_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094d1c6a392248899e99883ab8a9655b" is_pre_election: true
I20260809 03:29:22.302484 30436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebe255e5ab04e46a22d98130f4bb59e" candidate_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" is_pre_election: true
I20260809 03:29:22.302582 30293 raft_consensus.cc:2454] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f96c45e6cd1b4074ac6b19cb8d652025 in term 0.
I20260809 03:29:22.302794 30436 raft_consensus.cc:2454] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f96c45e6cd1b4074ac6b19cb8d652025 in term 0.
I20260809 03:29:22.303146 30514 leader_election.cc:302] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 094d1c6a392248899e99883ab8a9655b, f96c45e6cd1b4074ac6b19cb8d652025; no voters:
I20260809 03:29:22.303377 30662 raft_consensus.cc:2781] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:29:22.303470 30662 raft_consensus.cc:482] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:29:22.303526 30662 raft_consensus.cc:3037] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:22.304596 30662 raft_consensus.cc:504] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35417 } } peers { permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 } } peers { permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 } }
I20260809 03:29:22.304953 30662 leader_election.cc:288] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [CANDIDATE]: Term 1 election: Requested vote from peers 094d1c6a392248899e99883ab8a9655b (127.9.46.193:42721), dd0d4d2269f34f4e9ed1d188b2e00744 (127.9.46.194:37941)
I20260809 03:29:22.305368 30293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebe255e5ab04e46a22d98130f4bb59e" candidate_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094d1c6a392248899e99883ab8a9655b"
I20260809 03:29:22.305444 30436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ebe255e5ab04e46a22d98130f4bb59e" candidate_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744"
I20260809 03:29:22.305510 30293 raft_consensus.cc:3037] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:22.305574 30436 raft_consensus.cc:3037] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:22.306627 30436 raft_consensus.cc:2454] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f96c45e6cd1b4074ac6b19cb8d652025 in term 1.
I20260809 03:29:22.306679 30293 raft_consensus.cc:2454] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f96c45e6cd1b4074ac6b19cb8d652025 in term 1.
I20260809 03:29:22.306944 30514 leader_election.cc:302] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd0d4d2269f34f4e9ed1d188b2e00744, f96c45e6cd1b4074ac6b19cb8d652025; no voters:
I20260809 03:29:22.307117 30662 raft_consensus.cc:2781] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:22.307343 30662 raft_consensus.cc:686] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 1 LEADER]: Becoming Leader. State: Replica: f96c45e6cd1b4074ac6b19cb8d652025, State: Running, Role: LEADER
I20260809 03:29:22.307695 30662 consensus_queue.cc:237] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35417 } } peers { permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 } } peers { permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 } }
I20260809 03:29:22.310945 30139 catalog_manager.cc:5495] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 reported cstate change: term changed from 0 to 1, leader changed from <none> to f96c45e6cd1b4074ac6b19cb8d652025 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f96c45e6cd1b4074ac6b19cb8d652025" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 35417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 } health_report { overall_health: UNKNOWN } } }
I20260809 03:29:22.402496 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:22.399076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35768 (local address 127.9.46.195:35417)
0809 03:29:22.399220 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:22.399224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:22.399238 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:29:22.399312 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:22.399314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:22.399369 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:22.399458 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:22.399690 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.400180 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.401000 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.401248 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.401329 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:22.401662 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:22.402049 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:22.402138 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:22.402233 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 03:29:22.402292 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:29:22.408277 30436 raft_consensus.cc:1261] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744 [term 1 FOLLOWER]: Refusing update from remote peer f96c45e6cd1b4074ac6b19cb8d652025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:22.408277 30293 raft_consensus.cc:1261] T 4ebe255e5ab04e46a22d98130f4bb59e P 094d1c6a392248899e99883ab8a9655b [term 1 FOLLOWER]: Refusing update from remote peer f96c45e6cd1b4074ac6b19cb8d652025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:22.409031 30662 consensus_queue.cc:1035] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd0d4d2269f34f4e9ed1d188b2e00744" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 37941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:22.409292 30666 consensus_queue.cc:1035] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "094d1c6a392248899e99883ab8a9655b" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 42721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:22.417483 30670 mvcc.cc:205] Tried to move back new op lower bound from 7316464281217236992 to 7316464280815235072. Current Snapshot: MvccSnapshot[applied={T|T < 7316464281217236992}]
I20260809 03:29:22.417883 30669 mvcc.cc:205] Tried to move back new op lower bound from 7316464281217236992 to 7316464280815235072. Current Snapshot: MvccSnapshot[applied={T|T < 7316464281217236992}]
I20260809 03:29:22.427573 30140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:22.427722 30140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:22.430120 30140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ade63e80a1f847998a8ea5a678636885: Sending DeleteTablet for 3 replicas of tablet 4ebe255e5ab04e46a22d98130f4bb59e
I20260809 03:29:22.430893 30565 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ebe255e5ab04e46a22d98130f4bb59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:22 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:35754
I20260809 03:29:22.431131 30415 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ebe255e5ab04e46a22d98130f4bb59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:22 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:36868
I20260809 03:29:22.431157 30271 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ebe255e5ab04e46a22d98130f4bb59e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:22 UTC) from {username='kudu', principal='kudu/127.9.46.254@KRBTEST.COM'} at 127.0.0.1:39386
I20260809 03:29:22.431505 30679 tablet_replica.cc:330] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: stopping tablet replica
I20260809 03:29:22.431921 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 30205
I20260809 03:29:22.433327 30679 raft_consensus.cc:2229] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:29:22.433816 30679 raft_consensus.cc:2258] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:22.435361 30679 ts_tablet_manager.cc:1905] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:29:22.437744 30679 ts_tablet_manager.cc:1918] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:29:22.437853 30679 log.cc:1196] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/4ebe255e5ab04e46a22d98130f4bb59e
I20260809 03:29:22.438138 30679 ts_tablet_manager.cc:1939] T 4ebe255e5ab04e46a22d98130f4bb59e P f96c45e6cd1b4074ac6b19cb8d652025: Deleting consensus metadata
I20260809 03:29:22.439343 30126 catalog_manager.cc:4841] TS f96c45e6cd1b4074ac6b19cb8d652025 (127.9.46.195:35417): tablet 4ebe255e5ab04e46a22d98130f4bb59e (table test-table [id=e1508cdc614745fc8c3abee0064972da]) successfully deleted
I20260809 03:29:22.440752 30680 tablet_replica.cc:330] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744: stopping tablet replica
I20260809 03:29:22.441045 30680 raft_consensus.cc:2229] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:29:22.441362 30680 raft_consensus.cc:2258] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:22.442569 30680 ts_tablet_manager.cc:1905] T 4ebe255e5ab04e46a22d98130f4bb59e P dd0d4d2269f34f4e9ed1d188b2e00744: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 03:29:22.444435 30124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:42721 (error 108)
I20260809 03:29:22.444490 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 30348
I20260809 03:29:22.445650 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:22.445372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:42721 (local address 127.0.0.1:39394)
0809 03:29:22.445495 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:22.445578 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 03:29:22.445845 30124 catalog_manager.cc:4584] TS 094d1c6a392248899e99883ab8a9655b (127.9.46.193:42721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ebe255e5ab04e46a22d98130f4bb59e: Network error: Client connection negotiation failed: client connection to 127.9.46.193:42721: connect: Connection refused (error 111)
I20260809 03:29:22.456254 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 30494
I20260809 03:29:22.456339 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:22.456067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:37941 (local address 127.0.0.1:36872)
0809 03:29:22.456196 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:22.456249 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.194:37941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 03:29:22.462982 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 30106
2026-08-09T03:29:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2675 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T03:29:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:22Z Disabled control of system clock
I20260809 03:29:22.495404 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:33049
--webserver_interface=127.9.46.254
--webserver_port=0
--builtin_ntp_servers=127.9.46.212:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:33049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:29:22.605356 30686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:22.605594 30686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:22.605633 30686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:22.609985 30686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:22.610045 30686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:22.610062 30686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:22.610080 30686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:22.610095 30686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:22.614833 30686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:33049
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:33049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:22.615787 30686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:22.617085 30686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:22.625231 30692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:22.625249 30691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:22.625418 30686 server_base.cc:1034] running on GCE node
W20260809 03:29:22.625242 30694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:22.626075 30686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:22.627166 30686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:22.628361 30686 hybrid_clock.cc:648] HybridClock initialized: now 1786246162628333 us; error 40 us; skew 500 ppm
I20260809 03:29:22.630255 30686 webserver.cc:460] Webserver started at http://127.9.46.254:34803/ using document root <none> and password file <none>
I20260809 03:29:22.630844 30686 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:22.630935 30686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:22.631167 30686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:22.633253 30686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "5c15569c07ed4687bb1c716ae805c704"
format_stamp: "Formatted at 2026-08-09 03:29:22 on dist-test-slave-65mx"
server_key: "7768750d89591a377452f132f557a3e7"
server_key_iv: "39cf17b3d6292a608ec46384f45c0438"
server_key_version: "encryptionkey@0"
I20260809 03:29:22.633849 30686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "5c15569c07ed4687bb1c716ae805c704"
format_stamp: "Formatted at 2026-08-09 03:29:22 on dist-test-slave-65mx"
server_key: "7768750d89591a377452f132f557a3e7"
server_key_iv: "39cf17b3d6292a608ec46384f45c0438"
server_key_version: "encryptionkey@0"
I20260809 03:29:22.637444 30686 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:29:22.639806 30700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:22.641053 30686 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:22.641192 30686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "5c15569c07ed4687bb1c716ae805c704"
format_stamp: "Formatted at 2026-08-09 03:29:22 on dist-test-slave-65mx"
server_key: "7768750d89591a377452f132f557a3e7"
server_key_iv: "39cf17b3d6292a608ec46384f45c0438"
server_key_version: "encryptionkey@0"
I20260809 03:29:22.641301 30686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:22.689278 30686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:22.690037 30686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:22.690249 30686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:22.697175 30686 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:33049
I20260809 03:29:22.697182 30753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:33049 every 8 connection(s)
I20260809 03:29:22.698130 30686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:22.701239 30754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:22.701821 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 30686
I20260809 03:29:22.701964 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:22.702277 9403 external_mini_cluster.cc:1428] Setting key 5d425f27a373301d5e78db18df7d89cd
I20260809 03:29:22.707211 30754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704: Bootstrap starting.
I20260809 03:29:22.709467 30754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:22.710192 30754 log.cc:824] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:22.711578 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:22.703760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.9.46.254:33049)
0809 03:29:22.704421 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:22.704436 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:22.704472 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:29:22.705193 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:22.705211 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:22.705592 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:22.705915 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:22.706338 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.707260 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.708164 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.708689 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.708857 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:22.709795 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:22.709805 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:22.709817 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:22.710202 (+ 385us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:22.710256 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:22.710276 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:22.710394 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:22.710545 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 03:29:22.710829 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":136,"threads_started":1}
I20260809 03:29:22.712191 30754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704: No bootstrap required, opened a new log
I20260809 03:29:22.715595 30754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c15569c07ed4687bb1c716ae805c704" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33049 } }
I20260809 03:29:22.715838 30754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:22.715920 30754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c15569c07ed4687bb1c716ae805c704, State: Initialized, Role: FOLLOWER
I20260809 03:29:22.716368 30754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c15569c07ed4687bb1c716ae805c704" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33049 } }
I20260809 03:29:22.716513 30754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:22.716619 30754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:22.716720 30754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:22.717648 30754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c15569c07ed4687bb1c716ae805c704" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33049 } }
I20260809 03:29:22.717972 30754 leader_election.cc:302] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c15569c07ed4687bb1c716ae805c704; no voters:
I20260809 03:29:22.718320 30754 leader_election.cc:288] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:22.718421 30759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:22.718642 30759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [term 1 LEADER]: Becoming Leader. State: Replica: 5c15569c07ed4687bb1c716ae805c704, State: Running, Role: LEADER
I20260809 03:29:22.719007 30759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c15569c07ed4687bb1c716ae805c704" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33049 } }
I20260809 03:29:22.719527 30754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:22.720727 30760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c15569c07ed4687bb1c716ae805c704. Latest consensus state: current_term: 1 leader_uuid: "5c15569c07ed4687bb1c716ae805c704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c15569c07ed4687bb1c716ae805c704" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33049 } } }
I20260809 03:29:22.720860 30760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:22.721118 30761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c15569c07ed4687bb1c716ae805c704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c15569c07ed4687bb1c716ae805c704" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 33049 } } }
I20260809 03:29:22.721211 30761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:22.721946 30769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:22.724588 30769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:22.729832 30769 catalog_manager.cc:1347] Generated new cluster ID: 16d9a32790e345f285af667aa8b640bb
I20260809 03:29:22.729907 30769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:22.742455 30769 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:22.743359 30769 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:22.749320 30769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704: Generated new TSK 0
I20260809 03:29:22.749989 30769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:22.755589 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.193:0
--local_ip_for_outbound_sockets=127.9.46.193
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33049
--builtin_ntp_servers=127.9.46.212:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 03:29:22.869590 30782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:22.869870 30782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:22.869949 30782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:22.874020 30782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:22.874078 30782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:22.874166 30782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.193
I20260809 03:29:22.879019 30782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.46.193
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.193
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:22.880193 30782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:22.881678 30782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:22.891251 30792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:22.891242 30790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:22.891240 30789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:22.891691 30782 server_base.cc:1034] running on GCE node
I20260809 03:29:22.892117 30782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:22.892949 30782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:22.894245 30782 hybrid_clock.cc:648] HybridClock initialized: now 1786246162894230 us; error 24 us; skew 500 ppm
I20260809 03:29:22.896376 30782 webserver.cc:460] Webserver started at http://127.9.46.193:40961/ using document root <none> and password file <none>
I20260809 03:29:22.897082 30782 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:22.897184 30782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:22.897426 30782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:22.899618 30782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/data/instance:
uuid: "6903fcd269154f95becb0613d0eb03a0"
format_stamp: "Formatted at 2026-08-09 03:29:22 on dist-test-slave-65mx"
server_key: "ff04661650e3c438d8be68f415f7f5ca"
server_key_iv: "427d5ec179e1942191461f3c5327d7f6"
server_key_version: "encryptionkey@0"
I20260809 03:29:22.900240 30782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance:
uuid: "6903fcd269154f95becb0613d0eb03a0"
format_stamp: "Formatted at 2026-08-09 03:29:22 on dist-test-slave-65mx"
server_key: "ff04661650e3c438d8be68f415f7f5ca"
server_key_iv: "427d5ec179e1942191461f3c5327d7f6"
server_key_version: "encryptionkey@0"
I20260809 03:29:22.904256 30782 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 03:29:22.906922 30798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:22.908071 30782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:22.908214 30782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/wal
uuid: "6903fcd269154f95becb0613d0eb03a0"
format_stamp: "Formatted at 2026-08-09 03:29:22 on dist-test-slave-65mx"
server_key: "ff04661650e3c438d8be68f415f7f5ca"
server_key_iv: "427d5ec179e1942191461f3c5327d7f6"
server_key_version: "encryptionkey@0"
I20260809 03:29:22.908331 30782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:22.924010 30782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:22.924754 30782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:22.924950 30782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:22.925544 30782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:22.926501 30782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:22.926568 30782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:22.926662 30782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:22.926707 30782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:22.935649 30782 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.193:44423
I20260809 03:29:22.935669 30917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.193:44423 every 8 connection(s)
I20260809 03:29:22.936606 30782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/data/info.pb
I20260809 03:29:22.942929 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 30782
I20260809 03:29:22.943066 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-0/wal/instance
I20260809 03:29:22.943397 9403 external_mini_cluster.cc:1428] Setting key d52e4c3c7ac9ee12f29442de3fdddfe0
I20260809 03:29:22.945621 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.194:0
--local_ip_for_outbound_sockets=127.9.46.194
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33049
--builtin_ntp_servers=127.9.46.212:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:29:22.946610 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:22.938402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.193:58587 (local address 127.9.46.254:33049)
0809 03:29:22.938539 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:22.938543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:22.939572 (+ 1029us) server_negotiation.cc:407] Connection header received
0809 03:29:22.941008 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:22.941012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:22.941077 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:22.941175 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:22.942185 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.942745 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.944383 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.944774 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.944912 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:22.946105 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:22.946109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:22.946112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:22.946129 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:22.946147 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:22.946155 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:22.946244 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:22.946400 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 03:29:22.946470 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:29:22.947450 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:22.938792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:33049 (local address 127.9.46.193:58587)
0809 03:29:22.939419 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:22.939459 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:22.940806 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:22.941347 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:22.941353 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:22.941433 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:22.942027 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:22.942047 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.942884 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.942889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:22.944265 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:22.944275 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:22.944974 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:22.944978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:22.945180 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:22.945850 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:29:22.945852 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:22.945892 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:29:22.945954 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:22.946248 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:22.946260 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:22.946268 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:29:22.946397 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 03:29:22.946684 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":106,"threads_started":1}
I20260809 03:29:22.948781 30918 heartbeater.cc:347] Connected to a master server at 127.9.46.254:33049
I20260809 03:29:22.949013 30918 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:22.949600 30918 heartbeater.cc:511] Master 127.9.46.254:33049 requested a full tablet report, sending...
I20260809 03:29:22.951295 30718 ts_manager.cc:194] Registered new tserver with Master: 6903fcd269154f95becb0613d0eb03a0 (127.9.46.193:44423)
I20260809 03:29:22.952438 30718 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.193:58587
W20260809 03:29:23.057849 30921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:23.058096 30921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:23.058132 30921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:23.062042 30921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:23.062099 30921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:23.062184 30921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.194
I20260809 03:29:23.066813 30921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.46.194
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.194
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:23.067840 30921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:23.069195 30921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:23.078316 30929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:23.078336 30931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:23.078315 30927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:23.078626 30921 server_base.cc:1034] running on GCE node
I20260809 03:29:23.079077 30921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:23.079658 30921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:23.080837 30921 hybrid_clock.cc:648] HybridClock initialized: now 1786246163080822 us; error 33 us; skew 500 ppm
I20260809 03:29:23.082515 30921 webserver.cc:460] Webserver started at http://127.9.46.194:42845/ using document root <none> and password file <none>
I20260809 03:29:23.083117 30921 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:23.083187 30921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:23.083420 30921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:23.085714 30921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/data/instance:
uuid: "5ff595c8f8de46c58b104117729f367c"
format_stamp: "Formatted at 2026-08-09 03:29:23 on dist-test-slave-65mx"
server_key: "63213434575c93bddd706ce84f72b547"
server_key_iv: "0ff822c09c8fdfe014b1e0f39e449324"
server_key_version: "encryptionkey@0"
I20260809 03:29:23.086283 30921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance:
uuid: "5ff595c8f8de46c58b104117729f367c"
format_stamp: "Formatted at 2026-08-09 03:29:23 on dist-test-slave-65mx"
server_key: "63213434575c93bddd706ce84f72b547"
server_key_iv: "0ff822c09c8fdfe014b1e0f39e449324"
server_key_version: "encryptionkey@0"
I20260809 03:29:23.089926 30921 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:29:23.092276 30937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:23.093499 30921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:23.093624 30921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/wal
uuid: "5ff595c8f8de46c58b104117729f367c"
format_stamp: "Formatted at 2026-08-09 03:29:23 on dist-test-slave-65mx"
server_key: "63213434575c93bddd706ce84f72b547"
server_key_iv: "0ff822c09c8fdfe014b1e0f39e449324"
server_key_version: "encryptionkey@0"
I20260809 03:29:23.093734 30921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:23.110057 30921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:23.110808 30921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:23.110961 30921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:23.111529 30921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:23.112473 30921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:23.112520 30921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:23.112646 30921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:23.112673 30921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:23.121176 30921 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.194:46205
I20260809 03:29:23.121198 31054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.194:46205 every 8 connection(s)
I20260809 03:29:23.122179 30921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/data/info.pb
I20260809 03:29:23.123395 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 30921
I20260809 03:29:23.123540 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/wal/instance
I20260809 03:29:23.123893 9403 external_mini_cluster.cc:1428] Setting key 490b1e1e7d76b997f75a46c265589f6d
I20260809 03:29:23.125794 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.46.195:0
--local_ip_for_outbound_sockets=127.9.46.195
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33049
--builtin_ntp_servers=127.9.46.212:44449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 03:29:23.131316 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.124015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.194:58097 (local address 127.9.46.254:33049)
0809 03:29:23.124152 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:23.124157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:23.125292 (+ 1135us) server_negotiation.cc:407] Connection header received
0809 03:29:23.126732 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:23.126735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:23.126788 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:23.126865 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:23.127726 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.128246 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.129403 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.129627 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.129738 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.130839 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:23.130842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:23.130844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:23.130855 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:23.130874 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:23.130881 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:23.130957 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:23.131107 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 03:29:23.131174 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:29:23.132134 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.124419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:33049 (local address 127.9.46.194:58097)
0809 03:29:23.125134 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:23.125197 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:23.126528 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:23.127021 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:23.127028 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:23.127105 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:23.127571 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.127582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.128367 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.128370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.129278 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.129287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.129777 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.129781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.129940 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.130585 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:29:23.130586 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:23.130627 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:29:23.130688 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:23.130959 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:23.130971 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:23.130979 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 03:29:23.131106 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 03:29:23.131394 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":169,"threads_started":1}
I20260809 03:29:23.133303 31055 heartbeater.cc:347] Connected to a master server at 127.9.46.254:33049
I20260809 03:29:23.133566 31055 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:23.134232 31055 heartbeater.cc:511] Master 127.9.46.254:33049 requested a full tablet report, sending...
I20260809 03:29:23.135419 30718 ts_manager.cc:194] Registered new tserver with Master: 5ff595c8f8de46c58b104117729f367c (127.9.46.194:46205)
I20260809 03:29:23.136301 30718 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.194:58097
W20260809 03:29:23.238831 31058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:23.239089 31058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:23.239130 31058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:23.243184 31058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:23.243242 31058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:23.243296 31058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.46.195
I20260809 03:29:23.248018 31058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:44449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.46.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.46.195
--webserver_port=0
--tserver_master_addrs=127.9.46.254:33049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.46.195
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:23.249089 31058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:23.250489 31058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:23.259932 31063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:23.259925 31066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:23.259996 31058 server_base.cc:1034] running on GCE node
W20260809 03:29:23.259929 31064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:23.260512 31058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:23.261209 31058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:23.262384 31058 hybrid_clock.cc:648] HybridClock initialized: now 1786246163262371 us; error 24 us; skew 500 ppm
I20260809 03:29:23.264065 31058 webserver.cc:460] Webserver started at http://127.9.46.195:45333/ using document root <none> and password file <none>
I20260809 03:29:23.264667 31058 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:23.264758 31058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:23.264981 31058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:23.266952 31058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/data/instance:
uuid: "137f9d1a3e4d43498cfa7ea655554d87"
format_stamp: "Formatted at 2026-08-09 03:29:23 on dist-test-slave-65mx"
server_key: "9c285b502a7707314e23653808954a18"
server_key_iv: "04171d3e381782328c4b3a7348ba5d46"
server_key_version: "encryptionkey@0"
I20260809 03:29:23.267504 31058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance:
uuid: "137f9d1a3e4d43498cfa7ea655554d87"
format_stamp: "Formatted at 2026-08-09 03:29:23 on dist-test-slave-65mx"
server_key: "9c285b502a7707314e23653808954a18"
server_key_iv: "04171d3e381782328c4b3a7348ba5d46"
server_key_version: "encryptionkey@0"
I20260809 03:29:23.271016 31058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 03:29:23.273326 31072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:23.274405 31058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:23.274539 31058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/wal
uuid: "137f9d1a3e4d43498cfa7ea655554d87"
format_stamp: "Formatted at 2026-08-09 03:29:23 on dist-test-slave-65mx"
server_key: "9c285b502a7707314e23653808954a18"
server_key_iv: "04171d3e381782328c4b3a7348ba5d46"
server_key_version: "encryptionkey@0"
I20260809 03:29:23.274647 31058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:23.314038 31058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:23.314752 31058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:23.314951 31058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:23.315552 31058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 03:29:23.316498 31058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 03:29:23.316601 31058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:23.316689 31058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 03:29:23.316737 31058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:23.325706 31058 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.195:44139
I20260809 03:29:23.325745 31191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.195:44139 every 8 connection(s)
I20260809 03:29:23.326586 31058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/data/info.pb
I20260809 03:29:23.333132 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 31058
I20260809 03:29:23.333277 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/wal/instance
I20260809 03:29:23.333549 9403 external_mini_cluster.cc:1428] Setting key b602717a005d2d1b64094f1222bf6032
I20260809 03:29:23.335664 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.328303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:45611 (local address 127.9.46.254:33049)
0809 03:29:23.328450 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:23.328455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:23.329590 (+ 1135us) server_negotiation.cc:407] Connection header received
0809 03:29:23.330911 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:23.330914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:23.330962 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:23.331038 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:23.331887 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.332401 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.333637 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.333944 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.334064 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.335184 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:23.335187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:23.335188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:23.335200 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:23.335218 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:23.335226 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:23.335298 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:23.335464 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 03:29:23.335524 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:29:23.336484 31194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.328817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:33049 (local address 127.9.46.195:45611)
0809 03:29:23.329443 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:23.329484 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:23.330724 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:23.331195 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:23.331201 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:23.331282 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:23.331766 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.331778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.332503 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.332505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.333503 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.333512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.334105 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.334108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.334245 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.334917 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 03:29:23.334918 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:23.334957 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 03:29:23.335016 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:23.335311 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:23.335321 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:23.335327 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 03:29:23.335450 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 03:29:23.335728 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
I20260809 03:29:23.337554 31192 heartbeater.cc:347] Connected to a master server at 127.9.46.254:33049
I20260809 03:29:23.337801 31192 heartbeater.cc:464] Registering TS with master...
I20260809 03:29:23.338285 31192 heartbeater.cc:511] Master 127.9.46.254:33049 requested a full tablet report, sending...
I20260809 03:29:23.339239 30718 ts_manager.cc:194] Registered new tserver with Master: 137f9d1a3e4d43498cfa7ea655554d87 (127.9.46.195:44139)
I20260809 03:29:23.339795 30718 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.46.195:45611
I20260809 03:29:23.347256 9403 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 03:29:23.352604 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.349229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34506 (local address 127.9.46.254:33049)
0809 03:29:23.349346 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:23.349350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:23.349487 (+ 137us) server_negotiation.cc:407] Connection header received
0809 03:29:23.349586 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:23.349589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:23.349627 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:23.349697 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:23.349932 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.350426 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.351212 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.351409 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.351482 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.352240 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:23.352242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:23.352244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:23.352254 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:23.352269 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:23.352276 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:23.352327 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:23.352392 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 03:29:23.352437 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:29:23.355420 30718 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 03:29:23.358462 30718 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 03:29:23.372339 31206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.366305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44423 (local address 127.0.0.1:55472)
0809 03:29:23.366641 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:23.366688 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:23.366806 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:23.367576 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:23.367582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:23.367606 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:23.367768 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.367776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.369168 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.369171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.370634 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.370643 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.371776 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.371780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.371906 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.371949 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 03:29:23.372054 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 03:29:23.372184 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260809 03:29:23.372397 31207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.366455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55472 (local address 127.9.46.193:44423)
0809 03:29:23.366789 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:23.366799 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:23.366822 (+ 23us) server_negotiation.cc:407] Connection header received
0809 03:29:23.366995 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:23.367003 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:23.367214 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:23.367377 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:23.367883 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.369054 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.370788 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.371645 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.371760 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.371954 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:23.372110 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 03:29:23.372209 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":130,"threads_started":1}
I20260809 03:29:23.372634 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.367271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51210 (local address 127.9.46.194:46205)
0809 03:29:23.367679 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:23.367698 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:23.368129 (+ 431us) server_negotiation.cc:407] Connection header received
0809 03:29:23.368218 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:23.368224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:23.368388 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:23.368533 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:23.368787 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.369638 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.370916 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.371645 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.371760 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.371954 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:23.372317 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 03:29:23.372429 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":105,"threads_started":1}
I20260809 03:29:23.372529 31208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.366993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:46205 (local address 127.0.0.1:51210)
0809 03:29:23.367716 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:23.367729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:23.367823 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:23.368535 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:23.368556 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:23.368574 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:23.368669 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.368676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.369749 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.369754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.370781 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.370791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.372037 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.372041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.372158 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.372191 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 03:29:23.372270 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 03:29:23.372341 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260809 03:29:23.373018 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.367320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.195:44139 (local address 127.0.0.1:50560)
0809 03:29:23.368042 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:23.368056 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:23.368161 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:23.368736 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:23.368741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:23.368764 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:23.368842 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.368847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.370174 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.370177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.371140 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.371148 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.372037 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.372041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.372653 (+ 612us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.372694 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 03:29:23.372782 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 03:29:23.372852 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"mutex_wait_us":487,"thread_start_us":90,"threads_started":1}
I20260809 03:29:23.373550 31211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.367499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.9.46.195:44139)
0809 03:29:23.368059 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:23.368073 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:23.368182 (+ 109us) server_negotiation.cc:407] Connection header received
0809 03:29:23.368290 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:23.368294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:23.368424 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:23.368589 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:23.368951 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.370059 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.371248 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.371802 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.371909 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.372083 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:23.373166 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0809 03:29:23.373262 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":157,"threads_started":1}
I20260809 03:29:23.375571 30848 tablet_service.cc:1467] Processing CreateTablet for tablet 54d3c40d329840eb97f59f842908b614 (DEFAULT_TABLE table=test-table [id=d0395a663e4a4567bef92325296ae34d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:23.375694 30986 tablet_service.cc:1467] Processing CreateTablet for tablet 54d3c40d329840eb97f59f842908b614 (DEFAULT_TABLE table=test-table [id=d0395a663e4a4567bef92325296ae34d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:23.376113 31125 tablet_service.cc:1467] Processing CreateTablet for tablet 54d3c40d329840eb97f59f842908b614 (DEFAULT_TABLE table=test-table [id=d0395a663e4a4567bef92325296ae34d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 03:29:23.376626 30848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54d3c40d329840eb97f59f842908b614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:23.376770 30986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54d3c40d329840eb97f59f842908b614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:23.377061 31125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54d3c40d329840eb97f59f842908b614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:23.383015 31212 tablet_bootstrap.cc:492] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0: Bootstrap starting.
I20260809 03:29:23.384351 31213 tablet_bootstrap.cc:492] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: Bootstrap starting.
I20260809 03:29:23.384877 31214 tablet_bootstrap.cc:492] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: Bootstrap starting.
I20260809 03:29:23.385257 31212 tablet_bootstrap.cc:654] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:23.385993 31212 log.cc:824] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:23.386459 31213 tablet_bootstrap.cc:654] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:23.386878 31214 tablet_bootstrap.cc:654] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:23.387255 31213 log.cc:824] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:23.387661 31214 log.cc:824] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:23.388006 31212 tablet_bootstrap.cc:492] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0: No bootstrap required, opened a new log
I20260809 03:29:23.388240 31212 ts_tablet_manager.cc:1397] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:29:23.389065 31213 tablet_bootstrap.cc:492] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: No bootstrap required, opened a new log
I20260809 03:29:23.389256 31213 ts_tablet_manager.cc:1397] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 03:29:23.389650 31214 tablet_bootstrap.cc:492] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: No bootstrap required, opened a new log
I20260809 03:29:23.389822 31214 ts_tablet_manager.cc:1397] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 03:29:23.392243 31213 raft_consensus.cc:348] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 } } peers { permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 } } peers { permanent_uuid: "137f9d1a3e4d43498cfa7ea655554d87" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 44139 } }
I20260809 03:29:23.392472 31213 raft_consensus.cc:374] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:23.392544 31213 raft_consensus.cc:729] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ff595c8f8de46c58b104117729f367c, State: Initialized, Role: FOLLOWER
I20260809 03:29:23.392609 31212 raft_consensus.cc:348] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 } } peers { permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 } } peers { permanent_uuid: "137f9d1a3e4d43498cfa7ea655554d87" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 44139 } }
I20260809 03:29:23.392860 31212 raft_consensus.cc:374] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:23.392958 31212 raft_consensus.cc:729] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6903fcd269154f95becb0613d0eb03a0, State: Initialized, Role: FOLLOWER
I20260809 03:29:23.393028 31213 consensus_queue.cc:260] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 } } peers { permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 } } peers { permanent_uuid: "137f9d1a3e4d43498cfa7ea655554d87" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 44139 } }
I20260809 03:29:23.393373 31212 consensus_queue.cc:260] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 } } peers { permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 } } peers { permanent_uuid: "137f9d1a3e4d43498cfa7ea655554d87" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 44139 } }
I20260809 03:29:23.393748 31055 heartbeater.cc:503] Master 127.9.46.254:33049 was elected leader, sending a full tablet report...
I20260809 03:29:23.393880 31214 raft_consensus.cc:348] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 } } peers { permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 } } peers { permanent_uuid: "137f9d1a3e4d43498cfa7ea655554d87" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 44139 } }
I20260809 03:29:23.394099 31214 raft_consensus.cc:374] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:23.394098 31213 ts_tablet_manager.cc:1428] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 03:29:23.394142 31214 raft_consensus.cc:729] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 137f9d1a3e4d43498cfa7ea655554d87, State: Initialized, Role: FOLLOWER
I20260809 03:29:23.394366 31212 ts_tablet_manager.cc:1428] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 03:29:23.394685 30918 heartbeater.cc:503] Master 127.9.46.254:33049 was elected leader, sending a full tablet report...
I20260809 03:29:23.394820 31214 consensus_queue.cc:260] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 } } peers { permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 } } peers { permanent_uuid: "137f9d1a3e4d43498cfa7ea655554d87" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 44139 } }
I20260809 03:29:23.395720 31214 ts_tablet_manager.cc:1428] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 03:29:23.395817 31192 heartbeater.cc:503] Master 127.9.46.254:33049 was elected leader, sending a full tablet report...
I20260809 03:29:23.429152 31221 raft_consensus.cc:482] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 03:29:23.429370 31221 raft_consensus.cc:504] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 } } peers { permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 } } peers { permanent_uuid: "137f9d1a3e4d43498cfa7ea655554d87" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 44139 } }
I20260809 03:29:23.430461 31221 leader_election.cc:288] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6903fcd269154f95becb0613d0eb03a0 (127.9.46.193:44423), 5ff595c8f8de46c58b104117729f367c (127.9.46.194:46205)
I20260809 03:29:23.433962 31194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.430684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44423 (local address 127.9.46.195:58477)
0809 03:29:23.430806 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:23.430823 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:23.430916 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:23.431177 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:23.431180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:23.431200 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:23.431343 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.431350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.432076 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.432082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.432917 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.432926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.433540 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.433544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.433654 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.433672 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 03:29:23.433756 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 03:29:23.433827 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 03:29:23.433961 31207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.430775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:58477 (local address 127.9.46.193:44423)
0809 03:29:23.430913 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:23.430920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:23.430938 (+ 18us) server_negotiation.cc:407] Connection header received
0809 03:29:23.431003 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:23.431008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:23.431066 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:23.431163 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:23.431466 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.431967 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.433041 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.433426 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.433549 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.433600 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:23.433774 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 03:29:23.433844 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:29:23.434386 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.431116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.46.195:47399 (local address 127.9.46.194:46205)
0809 03:29:23.431249 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:23.431253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:23.431444 (+ 191us) server_negotiation.cc:407] Connection header received
0809 03:29:23.431587 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:23.431589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:23.431631 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:23.431707 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 03:29:23.431974 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.432447 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.433449 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.433846 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.433976 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.434022 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:23.434213 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 03:29:23.434280 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:29:23.434398 31222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.430958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.194:46205 (local address 127.9.46.195:47399)
0809 03:29:23.431359 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:23.431370 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:23.431479 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:23.431732 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:23.431735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:23.431750 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 03:29:23.431875 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.431880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.432580 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.432584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.433327 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:23.433335 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.433983 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.433987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:23.434085 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.434109 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 03:29:23.434198 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 03:29:23.434259 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":120,"threads_started":1}
I20260809 03:29:23.434655 30869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d3c40d329840eb97f59f842908b614" candidate_uuid: "137f9d1a3e4d43498cfa7ea655554d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6903fcd269154f95becb0613d0eb03a0" is_pre_election: true
I20260809 03:29:23.435005 30869 raft_consensus.cc:2454] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 137f9d1a3e4d43498cfa7ea655554d87 in term 0.
I20260809 03:29:23.435343 31008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d3c40d329840eb97f59f842908b614" candidate_uuid: "137f9d1a3e4d43498cfa7ea655554d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff595c8f8de46c58b104117729f367c" is_pre_election: true
I20260809 03:29:23.435529 31073 leader_election.cc:302] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 137f9d1a3e4d43498cfa7ea655554d87, 6903fcd269154f95becb0613d0eb03a0; no voters:
I20260809 03:29:23.435599 31008 raft_consensus.cc:2454] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 137f9d1a3e4d43498cfa7ea655554d87 in term 0.
I20260809 03:29:23.435803 31221 raft_consensus.cc:2781] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 03:29:23.435902 31221 raft_consensus.cc:482] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 03:29:23.435990 31221 raft_consensus.cc:3037] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:23.437125 31221 raft_consensus.cc:504] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 } } peers { permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 } } peers { permanent_uuid: "137f9d1a3e4d43498cfa7ea655554d87" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 44139 } }
I20260809 03:29:23.437453 31221 leader_election.cc:288] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [CANDIDATE]: Term 1 election: Requested vote from peers 6903fcd269154f95becb0613d0eb03a0 (127.9.46.193:44423), 5ff595c8f8de46c58b104117729f367c (127.9.46.194:46205)
W20260809 03:29:23.437744 30919 tablet.cc:2369] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:29:23.437845 30869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d3c40d329840eb97f59f842908b614" candidate_uuid: "137f9d1a3e4d43498cfa7ea655554d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6903fcd269154f95becb0613d0eb03a0"
I20260809 03:29:23.437899 31008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d3c40d329840eb97f59f842908b614" candidate_uuid: "137f9d1a3e4d43498cfa7ea655554d87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ff595c8f8de46c58b104117729f367c"
I20260809 03:29:23.437990 30869 raft_consensus.cc:3037] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:23.438017 31008 raft_consensus.cc:3037] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:23.439033 31008 raft_consensus.cc:2454] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 137f9d1a3e4d43498cfa7ea655554d87 in term 1.
I20260809 03:29:23.439101 30869 raft_consensus.cc:2454] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 137f9d1a3e4d43498cfa7ea655554d87 in term 1.
I20260809 03:29:23.439360 31073 leader_election.cc:302] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 137f9d1a3e4d43498cfa7ea655554d87, 5ff595c8f8de46c58b104117729f367c; no voters:
I20260809 03:29:23.439517 31221 raft_consensus.cc:2781] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:23.439759 31221 raft_consensus.cc:686] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 1 LEADER]: Becoming Leader. State: Replica: 137f9d1a3e4d43498cfa7ea655554d87, State: Running, Role: LEADER
I20260809 03:29:23.440096 31221 consensus_queue.cc:237] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 } } peers { permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 } } peers { permanent_uuid: "137f9d1a3e4d43498cfa7ea655554d87" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 44139 } }
I20260809 03:29:23.443446 30716 catalog_manager.cc:5495] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 137f9d1a3e4d43498cfa7ea655554d87 (127.9.46.195). New cstate: current_term: 1 leader_uuid: "137f9d1a3e4d43498cfa7ea655554d87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "137f9d1a3e4d43498cfa7ea655554d87" member_type: VOTER last_known_addr { host: "127.9.46.195" port: 44139 } health_report { overall_health: HEALTHY } } }
I20260809 03:29:23.470646 31211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.467178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50572 (local address 127.9.46.195:44139)
0809 03:29:23.467312 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:23.467317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:23.467331 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:29:23.467435 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:23.467437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:23.467487 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:23.467597 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:23.467853 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.468357 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.469263 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:23.469477 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:23.469586 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:23.469883 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:23.470279 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:23.470407 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:23.470452 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 03:29:23.470503 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:29:23.476133 31008 raft_consensus.cc:1261] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c [term 1 FOLLOWER]: Refusing update from remote peer 137f9d1a3e4d43498cfa7ea655554d87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:23.476133 30869 raft_consensus.cc:1261] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0 [term 1 FOLLOWER]: Refusing update from remote peer 137f9d1a3e4d43498cfa7ea655554d87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:23.476831 31223 consensus_queue.cc:1035] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ff595c8f8de46c58b104117729f367c" member_type: VOTER last_known_addr { host: "127.9.46.194" port: 46205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:23.477036 31221 consensus_queue.cc:1035] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6903fcd269154f95becb0613d0eb03a0" member_type: VOTER last_known_addr { host: "127.9.46.193" port: 44423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:23.484979 31229 mvcc.cc:205] Tried to move back new op lower bound from 7316464285591363584 to 7316464285453611008. Current Snapshot: MvccSnapshot[applied={T|T < 7316464285591363584}]
I20260809 03:29:23.485877 31228 mvcc.cc:205] Tried to move back new op lower bound from 7316464285591363584 to 7316464285453611008. Current Snapshot: MvccSnapshot[applied={T|T < 7316464285591363584}]
I20260809 03:29:23.491775 30718 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:23.491945 30718 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 03:29:23.494306 30718 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c15569c07ed4687bb1c716ae805c704: Sending DeleteTablet for 3 replicas of tablet 54d3c40d329840eb97f59f842908b614
I20260809 03:29:23.495126 30986 tablet_service.cc:1514] Processing DeleteTablet for tablet 54d3c40d329840eb97f59f842908b614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:23 UTC) from {username='slave'} at 127.0.0.1:51210
I20260809 03:29:23.495253 30848 tablet_service.cc:1514] Processing DeleteTablet for tablet 54d3c40d329840eb97f59f842908b614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:23 UTC) from {username='slave'} at 127.0.0.1:55472
I20260809 03:29:23.495313 31125 tablet_service.cc:1514] Processing DeleteTablet for tablet 54d3c40d329840eb97f59f842908b614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 03:29:23 UTC) from {username='slave'} at 127.0.0.1:50560
I20260809 03:29:23.495741 31236 tablet_replica.cc:330] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: stopping tablet replica
I20260809 03:29:23.496091 31238 tablet_replica.cc:330] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0: stopping tablet replica
I20260809 03:29:23.496179 31239 tablet_replica.cc:330] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: stopping tablet replica
I20260809 03:29:23.496207 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 30782
I20260809 03:29:23.496487 31239 raft_consensus.cc:2229] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 1 LEADER]: Raft consensus shutting down.
I20260809 03:29:23.496490 31238 raft_consensus.cc:2229] T 54d3c40d329840eb97f59f842908b614 P 6903fcd269154f95becb0613d0eb03a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:29:23.496881 31239 raft_consensus.cc:2258] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:23.497628 31236 raft_consensus.cc:2229] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 03:29:23.498015 31236 raft_consensus.cc:2258] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 03:29:23.498803 31239 ts_tablet_manager.cc:1905] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:29:23.499476 31236 ts_tablet_manager.cc:1905] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 03:29:23.501657 31239 ts_tablet_manager.cc:1918] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:29:23.501796 31239 log.cc:1196] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-2/wal/wals/54d3c40d329840eb97f59f842908b614
I20260809 03:29:23.502189 31239 ts_tablet_manager.cc:1939] T 54d3c40d329840eb97f59f842908b614 P 137f9d1a3e4d43498cfa7ea655554d87: Deleting consensus metadata
I20260809 03:29:23.502312 31236 ts_tablet_manager.cc:1918] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 03:29:23.502414 31236 log.cc:1196] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: Deleting WAL directory at /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786245904130257-9403-0/minicluster-data/ts-1/wal/wals/54d3c40d329840eb97f59f842908b614
I20260809 03:29:23.502727 31236 ts_tablet_manager.cc:1939] T 54d3c40d329840eb97f59f842908b614 P 5ff595c8f8de46c58b104117729f367c: Deleting consensus metadata
I20260809 03:29:23.503479 30701 catalog_manager.cc:4841] TS 137f9d1a3e4d43498cfa7ea655554d87 (127.9.46.195:44139): tablet 54d3c40d329840eb97f59f842908b614 (table test-table [id=d0395a663e4a4567bef92325296ae34d]) successfully deleted
I20260809 03:29:23.503800 30701 catalog_manager.cc:4841] TS 5ff595c8f8de46c58b104117729f367c (127.9.46.194:46205): tablet 54d3c40d329840eb97f59f842908b614 (table test-table [id=d0395a663e4a4567bef92325296ae34d]) successfully deleted
W20260809 03:29:23.506862 30701 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.46.193:44423 (error 108)
I20260809 03:29:23.507131 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 30921
I20260809 03:29:23.508124 31210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:23.507795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.193:44423 (local address 127.0.0.1:55480)
0809 03:29:23.507945 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:23.508027 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 03:29:23.509357 30701 catalog_manager.cc:4584] TS 6903fcd269154f95becb0613d0eb03a0 (127.9.46.193:44423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54d3c40d329840eb97f59f842908b614: Network error: Client connection negotiation failed: client connection to 127.9.46.193:44423: connect: Connection refused (error 111)
I20260809 03:29:23.515496 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 31058
I20260809 03:29:23.522255 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 30686
2026-08-09T03:29:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1058 ms)
[----------] 6 tests from EncryptionPolicyTest (11153 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:23 dist-test-slave-65mx krb5kdc[31244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:23 dist-test-slave-65mx krb5kdc[31244](info): set up 2 sockets
Aug 09 03:29:23 dist-test-slave-65mx krb5kdc[31244](info): commencing operation
krb5kdc: starting...
W20260809 03:29:24.625756 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:24 dist-test-slave-65mx krb5kdc[31244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:24.756735 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:24.868263 31257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:24.868634 31257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:24.868760 31257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:24.872816 31257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:24.872893 31257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:24.872931 31257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:24.872985 31257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:24.873025 31257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:24.878317 31257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42647
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:24.879403 31257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:24.880429 31257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:24.888908 31264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:24.888877 31263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:24.888998 31257 server_base.cc:1034] running on GCE node
W20260809 03:29:24.888877 31266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:24.889864 31257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:24.890908 31257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:24.892066 31257 hybrid_clock.cc:648] HybridClock initialized: now 1786246164892049 us; error 30 us; skew 500 ppm
Aug 09 03:29:24 dist-test-slave-65mx krb5kdc[31244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246164, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:24.894865 31257 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:24.895809 31257 webserver.cc:460] Webserver started at http://127.0.0.1:34107/ using document root <none> and password file <none>
I20260809 03:29:24.896325 31257 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:24.896389 31257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:24.896615 31257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:24.898617 31257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "e925bdc700aa4076998c79e71fff2542"
format_stamp: "Formatted at 2026-08-09 03:29:24 on dist-test-slave-65mx"
server_key: "86ceaf8abd9f3068e608b81eb7f9d8b1"
server_key_iv: "111837ca2325614706ca11290f59a96d"
server_key_version: "encryptionkey@0"
I20260809 03:29:24.899129 31257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "e925bdc700aa4076998c79e71fff2542"
format_stamp: "Formatted at 2026-08-09 03:29:24 on dist-test-slave-65mx"
server_key: "86ceaf8abd9f3068e608b81eb7f9d8b1"
server_key_iv: "111837ca2325614706ca11290f59a96d"
server_key_version: "encryptionkey@0"
I20260809 03:29:24.902685 31257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:29:24.905019 31274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:24.906186 31257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:24.906330 31257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "e925bdc700aa4076998c79e71fff2542"
format_stamp: "Formatted at 2026-08-09 03:29:24 on dist-test-slave-65mx"
server_key: "86ceaf8abd9f3068e608b81eb7f9d8b1"
server_key_iv: "111837ca2325614706ca11290f59a96d"
server_key_version: "encryptionkey@0"
I20260809 03:29:24.906440 31257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:24.938298 31257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:24.941365 31257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:24.941635 31257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:24.949144 31257 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42647
I20260809 03:29:24.949178 31327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42647 every 8 connection(s)
I20260809 03:29:24.950106 31257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:24.952868 31328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:24.953702 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 31257
I20260809 03:29:24.953835 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:24.954110 9403 external_mini_cluster.cc:1428] Setting key ace485a097b51a42cc2292349dd3f29b
I20260809 03:29:24.958771 31328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542: Bootstrap starting.
I20260809 03:29:24.961071 31328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:24.962078 31328 log.cc:824] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:29:24 dist-test-slave-65mx krb5kdc[31244](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:29:24.964304 31328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542: No bootstrap required, opened a new log
I20260809 03:29:24.968452 31328 raft_consensus.cc:348] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e925bdc700aa4076998c79e71fff2542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42647 } }
I20260809 03:29:24.968743 31328 raft_consensus.cc:374] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:24.968753 31331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:24.955508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52031 (local address 127.0.0.1:42647)
0809 03:29:24.955924 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:24.955939 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:24.955973 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:29:24.956653 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:24.956677 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:24.957008 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:24.957323 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:24.958154 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:24.959159 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:24.960007 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:24.960489 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:24.960714 (+ 225us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:24.963755 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:24.963760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:24.963772 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:24.963798 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:24.965638 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:24.966264 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:24.966268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:24.966273 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:24.966380 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:24.966743 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:24.966745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:24.966746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:24.967123 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:24.967290 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:24.967627 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 03:29:24.967873 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":136,"threads_started":1}
I20260809 03:29:24.968837 31328 raft_consensus.cc:729] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e925bdc700aa4076998c79e71fff2542, State: Initialized, Role: FOLLOWER
I20260809 03:29:24.969302 31328 consensus_queue.cc:260] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e925bdc700aa4076998c79e71fff2542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42647 } }
I20260809 03:29:24.969451 31328 raft_consensus.cc:388] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:24.969535 31328 raft_consensus.cc:482] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:24.969648 31328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:24.970844 31328 raft_consensus.cc:504] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e925bdc700aa4076998c79e71fff2542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42647 } }
I20260809 03:29:24.971216 31328 leader_election.cc:302] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e925bdc700aa4076998c79e71fff2542; no voters:
I20260809 03:29:24.971601 31328 leader_election.cc:288] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:24.971679 31333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:24.971933 31333 raft_consensus.cc:686] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [term 1 LEADER]: Becoming Leader. State: Replica: e925bdc700aa4076998c79e71fff2542, State: Running, Role: LEADER
I20260809 03:29:24.972030 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:24.972316 31333 consensus_queue.cc:237] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e925bdc700aa4076998c79e71fff2542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42647 } }
I20260809 03:29:24.972978 31328 sys_catalog.cc:564] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:24.973769 31334 sys_catalog.cc:455] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e925bdc700aa4076998c79e71fff2542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e925bdc700aa4076998c79e71fff2542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42647 } } }
I20260809 03:29:24.973876 31334 sys_catalog.cc:458] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:24.974200 31335 sys_catalog.cc:455] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e925bdc700aa4076998c79e71fff2542. Latest consensus state: current_term: 1 leader_uuid: "e925bdc700aa4076998c79e71fff2542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e925bdc700aa4076998c79e71fff2542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42647 } } }
I20260809 03:29:24.974305 31335 sys_catalog.cc:458] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:24.974630 31347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:24.977941 31347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:24.983794 31347 catalog_manager.cc:1347] Generated new cluster ID: f8a9969edd6147c5b8074d978a89eb07
I20260809 03:29:24.983911 31347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:24.984818 31331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:24.975639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57097 (local address 127.0.0.1:42647)
0809 03:29:24.975767 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:24.975773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:24.976019 (+ 246us) server_negotiation.cc:407] Connection header received
0809 03:29:24.976154 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:24.976157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:24.976207 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:24.976287 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:24.977245 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:24.977758 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:24.978807 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:24.979119 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:24.979230 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:24.980838 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:24.980842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:24.980845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:24.980870 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:24.982627 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:24.983265 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:24.983268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:24.983270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:24.983354 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:24.983777 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:24.983780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:24.983782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:24.984064 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:24.984211 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:24.984522 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 03:29:24.984652 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:29:24.996222 31347 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:24.997195 31347 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:25.009207 31347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e925bdc700aa4076998c79e71fff2542: Generated new TSK 0
I20260809 03:29:25.010033 31347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:25.020740 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 31257
W20260809 03:29:25.028438 31241 connection.cc:535] client connection to 127.0.0.1:42647 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42647): Connection reset by peer (error 104)
2026-08-09T03:29:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:25 dist-test-slave-65mx krb5kdc[31361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:25 dist-test-slave-65mx krb5kdc[31361](info): set up 2 sockets
Aug 09 03:29:25 dist-test-slave-65mx krb5kdc[31361](info): commencing operation
krb5kdc: starting...
W20260809 03:29:26.101169 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:26 dist-test-slave-65mx krb5kdc[31361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:26.235208 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33813
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:26.344817 31374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:26.345105 31374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:26.345202 31374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:26.349190 31374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:26.349268 31374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:26.349306 31374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:26.349357 31374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:26.349396 31374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:26.354583 31374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33813
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:26.355659 31374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:26.356691 31374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:26.364842 31383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:26.364871 31381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:26.364899 31380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:26.365036 31374 server_base.cc:1034] running on GCE node
I20260809 03:29:26.365669 31374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:26.366708 31374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:26.367895 31374 hybrid_clock.cc:648] HybridClock initialized: now 1786246166367866 us; error 37 us; skew 500 ppm
Aug 09 03:29:26 dist-test-slave-65mx krb5kdc[31361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246166, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:26.371147 31374 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:26.372112 31374 webserver.cc:460] Webserver started at http://127.0.0.1:36869/ using document root <none> and password file <none>
I20260809 03:29:26.372807 31374 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:26.372910 31374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:26.373174 31374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:26.375411 31374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6"
format_stamp: "Formatted at 2026-08-09 03:29:26 on dist-test-slave-65mx"
server_key: "d0e69d7f6320538e5eee712b9c1458f9"
server_key_iv: "63f0e8ceb0ea3a9c951a68f545d23dfa"
server_key_version: "encryptionkey@0"
I20260809 03:29:26.376078 31374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6"
format_stamp: "Formatted at 2026-08-09 03:29:26 on dist-test-slave-65mx"
server_key: "d0e69d7f6320538e5eee712b9c1458f9"
server_key_iv: "63f0e8ceb0ea3a9c951a68f545d23dfa"
server_key_version: "encryptionkey@0"
I20260809 03:29:26.380285 31374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 03:29:26.383015 31390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:26.384215 31374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:26.384366 31374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6"
format_stamp: "Formatted at 2026-08-09 03:29:26 on dist-test-slave-65mx"
server_key: "d0e69d7f6320538e5eee712b9c1458f9"
server_key_iv: "63f0e8ceb0ea3a9c951a68f545d23dfa"
server_key_version: "encryptionkey@0"
I20260809 03:29:26.384479 31374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:26.408907 31374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:26.412421 31374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:26.412658 31374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:26.419875 31374 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33813
I20260809 03:29:26.419905 31445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33813 every 8 connection(s)
I20260809 03:29:26.420835 31374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:26.422019 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 31374
I20260809 03:29:26.422165 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:26.422462 9403 external_mini_cluster.cc:1428] Setting key faccb755490a79a474c45b01b63e72d3
I20260809 03:29:26.423815 31446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:26.430687 31446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6: Bootstrap starting.
I20260809 03:29:26.432579 31449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:26.423833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48733 (local address 127.0.0.1:33813)
0809 03:29:26.424290 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:26.424309 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:26.424348 (+ 39us) server_negotiation.cc:407] Connection header received
0809 03:29:26.425212 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:26.425235 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:26.425641 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:26.425988 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:26.426392 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:26.427879 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:26.428824 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:26.429523 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:26.429681 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:26.430535 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:26.430541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:26.430557 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:26.431060 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:26.431129 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:26.431157 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:26.431297 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:26.431473 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 03:29:26.431599 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":137,"threads_started":1}
I20260809 03:29:26.433430 31446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:26.434165 31446 log.cc:824] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:26.435542 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:26.436450 31446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6: No bootstrap required, opened a new log
I20260809 03:29:26.440593 31446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33813 } }
I20260809 03:29:26.440898 31446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:26.440968 31446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ac0ae84a91405ba3ad6afb56f1e9b6, State: Initialized, Role: FOLLOWER
I20260809 03:29:26.441448 31446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33813 } }
I20260809 03:29:26.441645 31446 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:26.441721 31446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:26.441828 31446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:26.442824 31449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:26.437917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47993 (local address 127.0.0.1:33813)
0809 03:29:26.438043 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:26.438048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:26.438289 (+ 241us) server_negotiation.cc:407] Connection header received
0809 03:29:26.438368 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:26.438371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:26.438419 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:26.438509 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:26.439053 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:26.439841 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:26.440671 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:26.440995 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:26.441113 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:26.442260 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:26.442264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:26.442266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:26.442278 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:26.442305 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:26.442317 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:26.442400 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:26.442587 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 03:29:26.442648 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:29:26.443065 31446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33813 } }
I20260809 03:29:26.443405 31446 leader_election.cc:302] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87ac0ae84a91405ba3ad6afb56f1e9b6; no voters:
I20260809 03:29:26.443781 31446 leader_election.cc:288] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:26.443882 31456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:26.444114 31456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [term 1 LEADER]: Becoming Leader. State: Replica: 87ac0ae84a91405ba3ad6afb56f1e9b6, State: Running, Role: LEADER
I20260809 03:29:26.444480 31456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33813 } }
I20260809 03:29:26.445173 31446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:26.446165 31457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87ac0ae84a91405ba3ad6afb56f1e9b6. Latest consensus state: current_term: 1 leader_uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33813 } } }
I20260809 03:29:26.446311 31457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:26.446527 31458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ac0ae84a91405ba3ad6afb56f1e9b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33813 } } }
I20260809 03:29:26.446681 31458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:26.446971 31465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:26.449761 31465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:26.455302 31465 catalog_manager.cc:1347] Generated new cluster ID: 0181d4a1059e4e188f01f24c03e7754f
I20260809 03:29:26.455379 31465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:26.476310 31465 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:26.477305 31465 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:26.487380 31465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87ac0ae84a91405ba3ad6afb56f1e9b6: Generated new TSK 0
I20260809 03:29:26.488107 31465 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:26.493424 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 31374
W20260809 03:29:26.500475 31359 connection.cc:535] client connection to 127.0.0.1:33813 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33813): Connection reset by peer (error 104)
2026-08-09T03:29:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1471 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:26 dist-test-slave-65mx krb5kdc[31478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:26 dist-test-slave-65mx krb5kdc[31478](info): set up 2 sockets
Aug 09 03:29:26 dist-test-slave-65mx krb5kdc[31478](info): commencing operation
krb5kdc: starting...
W20260809 03:29:27.569152 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:27 dist-test-slave-65mx krb5kdc[31478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:27.696189 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37403
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:27.807024 31492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:27.807298 31492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:27.807340 31492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:27.811380 31492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:27.811443 31492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:27.811463 31492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:27.811502 31492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:27.811529 31492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:27.816774 31492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37403
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:27.817741 31492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:27.818663 31492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:27.827116 31497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:27.827175 31492 server_base.cc:1034] running on GCE node
W20260809 03:29:27.827193 31500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:27.827345 31498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:27.828011 31492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:27.829576 31492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:27.830786 31492 hybrid_clock.cc:648] HybridClock initialized: now 1786246167830726 us; error 81 us; skew 500 ppm
Aug 09 03:29:27 dist-test-slave-65mx krb5kdc[31478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:27.838526 31492 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:27.839457 31492 webserver.cc:460] Webserver started at http://127.0.0.1:34185/ using document root <none> and password file <none>
I20260809 03:29:27.839993 31492 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:27.840055 31492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:27.840219 31492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:27.842271 31492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "f82ed0233b0b42e8b456d3fc38808b37"
format_stamp: "Formatted at 2026-08-09 03:29:27 on dist-test-slave-65mx"
server_key: "92735849431b7749dd91a11a30c207d1"
server_key_iv: "8729eb456f5ea45862b4105203259fda"
server_key_version: "encryptionkey@0"
I20260809 03:29:27.842785 31492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "f82ed0233b0b42e8b456d3fc38808b37"
format_stamp: "Formatted at 2026-08-09 03:29:27 on dist-test-slave-65mx"
server_key: "92735849431b7749dd91a11a30c207d1"
server_key_iv: "8729eb456f5ea45862b4105203259fda"
server_key_version: "encryptionkey@0"
I20260809 03:29:27.846431 31492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 03:29:27.848786 31507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:27.850183 31492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:27.850337 31492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "f82ed0233b0b42e8b456d3fc38808b37"
format_stamp: "Formatted at 2026-08-09 03:29:27 on dist-test-slave-65mx"
server_key: "92735849431b7749dd91a11a30c207d1"
server_key_iv: "8729eb456f5ea45862b4105203259fda"
server_key_version: "encryptionkey@0"
I20260809 03:29:27.850449 31492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:27.855588 31492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:27.858721 31492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:27.858927 31492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:27.866462 31492 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37403
I20260809 03:29:27.866470 31560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37403 every 8 connection(s)
I20260809 03:29:27.867455 31492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:27.870187 31561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:27.872654 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 31492
I20260809 03:29:27.872812 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:27.873106 9403 external_mini_cluster.cc:1428] Setting key b859726369315d63f7bb8b301ae82dfb
I20260809 03:29:27.876386 31561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37: Bootstrap starting.
I20260809 03:29:27.879065 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:27.874512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56391 (local address 127.0.0.1:37403)
0809 03:29:27.874937 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:27.874956 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:27.874997 (+ 41us) server_negotiation.cc:407] Connection header received
0809 03:29:27.875806 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:27.875828 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:27.876180 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:27.876563 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:27.877493 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:27.877500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:27.877517 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:27.877532 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:27.877602 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:27.877634 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:27.877782 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:27.877892 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 03:29:27.878197 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":116,"threads_started":1}
I20260809 03:29:27.879117 31561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:27.879972 31561 log.cc:824] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:27.882057 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:27.882150 31561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37: No bootstrap required, opened a new log
I20260809 03:29:27.885897 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:27.884295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46629 (local address 127.0.0.1:37403)
0809 03:29:27.884436 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:27.884440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:27.884455 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:29:27.884618 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:27.884621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:27.884651 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:27.884736 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:27.885489 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:27.885492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:27.885495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:27.885531 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:27.885560 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:27.885571 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:27.885661 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:27.885739 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 03:29:27.885778 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 03:29:27.886201 31561 raft_consensus.cc:348] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82ed0233b0b42e8b456d3fc38808b37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } }
I20260809 03:29:27.886469 31561 raft_consensus.cc:374] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:27.886545 31561 raft_consensus.cc:729] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f82ed0233b0b42e8b456d3fc38808b37, State: Initialized, Role: FOLLOWER
I20260809 03:29:27.887030 31561 consensus_queue.cc:260] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82ed0233b0b42e8b456d3fc38808b37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } }
I20260809 03:29:27.887190 31561 raft_consensus.cc:388] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:27.887261 31561 raft_consensus.cc:482] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:27.887363 31561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:27.888289 31561 raft_consensus.cc:504] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82ed0233b0b42e8b456d3fc38808b37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } }
I20260809 03:29:27.888680 31561 leader_election.cc:302] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f82ed0233b0b42e8b456d3fc38808b37; no voters:
I20260809 03:29:27.888978 31561 leader_election.cc:288] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:27.889120 31571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:27.889355 31571 raft_consensus.cc:686] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [term 1 LEADER]: Becoming Leader. State: Replica: f82ed0233b0b42e8b456d3fc38808b37, State: Running, Role: LEADER
I20260809 03:29:27.889755 31571 consensus_queue.cc:237] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82ed0233b0b42e8b456d3fc38808b37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } }
I20260809 03:29:27.890246 31561 sys_catalog.cc:564] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:27.890882 31571 sys_catalog.cc:455] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f82ed0233b0b42e8b456d3fc38808b37. Latest consensus state: current_term: 1 leader_uuid: "f82ed0233b0b42e8b456d3fc38808b37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82ed0233b0b42e8b456d3fc38808b37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } } }
I20260809 03:29:27.891021 31571 sys_catalog.cc:458] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:27.891389 31572 sys_catalog.cc:455] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f82ed0233b0b42e8b456d3fc38808b37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82ed0233b0b42e8b456d3fc38808b37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } } }
I20260809 03:29:27.891462 31578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:27.891541 31572 sys_catalog.cc:458] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:27.894944 31578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:27.900310 31578 catalog_manager.cc:1347] Generated new cluster ID: 75ca01cf26474267a9ad279f35238617
I20260809 03:29:27.900385 31578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:27.924180 31578 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:27.924525 31578 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:27.931381 31578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f82ed0233b0b42e8b456d3fc38808b37: Generated new TSK 0
I20260809 03:29:27.932025 31578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:27.937814 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 31492
2026-08-09T03:29:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:27 dist-test-slave-65mx krb5kdc[31593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:27 dist-test-slave-65mx krb5kdc[31593](info): set up 2 sockets
Aug 09 03:29:27 dist-test-slave-65mx krb5kdc[31593](info): commencing operation
krb5kdc: starting...
W20260809 03:29:29.045173 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:29 dist-test-slave-65mx krb5kdc[31593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:29.175421 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:29.286019 31608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:29.286329 31608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:29.286417 31608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:29.290468 31608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:29.290550 31608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:29.290589 31608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:29.290647 31608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:29.290688 31608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:29.295835 31608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:29.297004 31608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:29.298110 31608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:29.306880 31613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:29.306881 31614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:29.306921 31608 server_base.cc:1034] running on GCE node
W20260809 03:29:29.306881 31616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:29.307717 31608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:29.308885 31608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:29.310046 31608 hybrid_clock.cc:648] HybridClock initialized: now 1786246169310017 us; error 44 us; skew 500 ppm
Aug 09 03:29:29 dist-test-slave-65mx krb5kdc[31593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:29.312927 31608 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:29.313782 31608 webserver.cc:460] Webserver started at http://127.0.0.1:39339/ using document root <none> and password file <none>
I20260809 03:29:29.314335 31608 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:29.314430 31608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:29.314666 31608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:29.316711 31608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4"
format_stamp: "Formatted at 2026-08-09 03:29:29 on dist-test-slave-65mx"
server_key: "8067b8b50a4208f9fe624273985792cf"
server_key_iv: "555335326f8f18b3626818acd5899cb4"
server_key_version: "encryptionkey@0"
I20260809 03:29:29.317312 31608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4"
format_stamp: "Formatted at 2026-08-09 03:29:29 on dist-test-slave-65mx"
server_key: "8067b8b50a4208f9fe624273985792cf"
server_key_iv: "555335326f8f18b3626818acd5899cb4"
server_key_version: "encryptionkey@0"
I20260809 03:29:29.320948 31608 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 03:29:29.323437 31624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:29.324702 31608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:29.324839 31608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4"
format_stamp: "Formatted at 2026-08-09 03:29:29 on dist-test-slave-65mx"
server_key: "8067b8b50a4208f9fe624273985792cf"
server_key_iv: "555335326f8f18b3626818acd5899cb4"
server_key_version: "encryptionkey@0"
I20260809 03:29:29.324954 31608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:29.344700 31608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:29.347702 31608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:29.347924 31608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:29.355370 31608 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39115
I20260809 03:29:29.355414 31677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39115 every 8 connection(s)
I20260809 03:29:29.356348 31608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:29.359146 31678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:29.362462 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 31608
I20260809 03:29:29.362620 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:29.362905 9403 external_mini_cluster.cc:1428] Setting key aa4d929f206822d3d4486859b27db8e5
I20260809 03:29:29.365533 31678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4: Bootstrap starting.
I20260809 03:29:29.367766 31678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:29.368521 31678 log.cc:824] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:29.370381 31678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4: No bootstrap required, opened a new log
Aug 09 03:29:29 dist-test-slave-65mx krb5kdc[31593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:29:29.374629 31678 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } }
I20260809 03:29:29.374842 31678 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:29.374950 31678 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd01a3d2d3d340f4bc6c60664b2e6ae4, State: Initialized, Role: FOLLOWER
I20260809 03:29:29.375448 31678 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } }
I20260809 03:29:29.375640 31678 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:29.375720 31678 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:29.375859 31678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:29.377055 31678 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } }
I20260809 03:29:29.377389 31678 leader_election.cc:302] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd01a3d2d3d340f4bc6c60664b2e6ae4; no voters:
I20260809 03:29:29.377750 31678 leader_election.cc:288] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:29.377837 31683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:29.378053 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:29.364373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52055 (local address 127.0.0.1:39115)
0809 03:29:29.365059 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:29.365082 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:29.365137 (+ 55us) server_negotiation.cc:407] Connection header received
0809 03:29:29.365997 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:29.366016 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:29.366363 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:29.366708 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:29.367544 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:29.368702 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:29.369604 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:29.370074 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:29.370246 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:29.373031 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:29.373036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:29.373048 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:29.373098 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:29.374789 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:29.375334 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:29.375338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:29.375342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:29.375451 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:29.375755 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:29.375758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:29.375762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:29.376234 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:29.376393 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:29.376743 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 03:29:29.377067 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":166,"threads_started":1}
I20260809 03:29:29.378058 31683 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [term 1 LEADER]: Becoming Leader. State: Replica: fd01a3d2d3d340f4bc6c60664b2e6ae4, State: Running, Role: LEADER
I20260809 03:29:29.378674 31683 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } }
I20260809 03:29:29.379093 31678 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:29.380350 31685 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd01a3d2d3d340f4bc6c60664b2e6ae4. Latest consensus state: current_term: 1 leader_uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } } }
I20260809 03:29:29.380476 31685 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:29.380904 31684 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd01a3d2d3d340f4bc6c60664b2e6ae4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39115 } } }
I20260809 03:29:29.381054 31684 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:29.381721 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 03:29:29.384758 31698 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:29:29.384876 31698 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:29:29.384953 31692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:29.385854 31692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:29.391594 31692 catalog_manager.cc:1347] Generated new cluster ID: d736974e49c343a2ab37cc4760f4241e
I20260809 03:29:29.391662 31692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:29.393976 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:29.384272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45467 (local address 127.0.0.1:39115)
0809 03:29:29.384403 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:29.384407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:29.384461 (+ 54us) server_negotiation.cc:407] Connection header received
0809 03:29:29.384627 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:29.384630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:29.384681 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:29.384770 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:29.385634 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:29.386506 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:29.387591 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:29.387840 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:29.387953 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:29.389353 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:29.389357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:29.389360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:29.389384 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:29.391028 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:29.392677 (+ 1649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:29.392681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:29.392683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:29.392782 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:29.393083 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:29.393086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:29.393088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:29.393306 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:29.393440 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:29.393679 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 03:29:29.393810 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:29:29.403558 31692 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:29.404451 31692 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:29.415961 31692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd01a3d2d3d340f4bc6c60664b2e6ae4: Generated new TSK 0
I20260809 03:29:29.416862 31692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:29.420439 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 31608
2026-08-09T03:29:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:29 dist-test-slave-65mx krb5kdc[31709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:29 dist-test-slave-65mx krb5kdc[31709](info): set up 2 sockets
Aug 09 03:29:29 dist-test-slave-65mx krb5kdc[31709](info): commencing operation
krb5kdc: starting...
W20260809 03:29:30.501266 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:30 dist-test-slave-65mx krb5kdc[31709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:30.632194 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46581
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:30.744602 31722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:30.744870 31722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:30.744912 31722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:30.749028 31722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:30.749087 31722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:30.749112 31722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:30.749163 31722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:30.749199 31722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:30.754447 31722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:30.755535 31722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:30.756583 31722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:30.764930 31732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:30.764994 31730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:30.765086 31728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:30.764966 31722 server_base.cc:1034] running on GCE node
I20260809 03:29:30.765625 31722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:30.766671 31722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:30.767832 31722 hybrid_clock.cc:648] HybridClock initialized: now 1786246170767802 us; error 43 us; skew 500 ppm
Aug 09 03:29:30 dist-test-slave-65mx krb5kdc[31709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246170, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:30.770516 31722 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:30.771309 31722 webserver.cc:460] Webserver started at http://127.0.0.1:35249/ using document root <none> and password file <none>
I20260809 03:29:30.771811 31722 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:30.771871 31722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:30.772032 31722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:30.774082 31722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "54151954d35b4c9d843cd8f566829fd4"
format_stamp: "Formatted at 2026-08-09 03:29:30 on dist-test-slave-65mx"
server_key: "4d4aabbaa5f819d0d2e6284a60d17c21"
server_key_iv: "900814c1446d7187eda2076519df9a12"
server_key_version: "encryptionkey@0"
I20260809 03:29:30.774596 31722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "54151954d35b4c9d843cd8f566829fd4"
format_stamp: "Formatted at 2026-08-09 03:29:30 on dist-test-slave-65mx"
server_key: "4d4aabbaa5f819d0d2e6284a60d17c21"
server_key_iv: "900814c1446d7187eda2076519df9a12"
server_key_version: "encryptionkey@0"
I20260809 03:29:30.778200 31722 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260809 03:29:30.780500 31740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:30.781731 31722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:29:30.781831 31722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "54151954d35b4c9d843cd8f566829fd4"
format_stamp: "Formatted at 2026-08-09 03:29:30 on dist-test-slave-65mx"
server_key: "4d4aabbaa5f819d0d2e6284a60d17c21"
server_key_iv: "900814c1446d7187eda2076519df9a12"
server_key_version: "encryptionkey@0"
I20260809 03:29:30.781921 31722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:30.798189 31722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:30.801614 31722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:30.801775 31722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:30.809046 31722 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46581
I20260809 03:29:30.809273 31794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46581 every 8 connection(s)
I20260809 03:29:30.810112 31722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:30.813056 31795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:30.818656 31795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4: Bootstrap starting.
I20260809 03:29:30.819638 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 31722
I20260809 03:29:30.819821 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:30.820089 9403 external_mini_cluster.cc:1428] Setting key 676081908fd233faf8cc02604afb560b
I20260809 03:29:30.821408 31795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:30.822348 31795 log.cc:824] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:30.824537 31795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4: No bootstrap required, opened a new log
I20260809 03:29:30.828708 31795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54151954d35b4c9d843cd8f566829fd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } }
I20260809 03:29:30.828977 31795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:30.829049 31795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54151954d35b4c9d843cd8f566829fd4, State: Initialized, Role: FOLLOWER
I20260809 03:29:30.829536 31795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54151954d35b4c9d843cd8f566829fd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } }
I20260809 03:29:30.829686 31795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:30.829753 31795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:30.829844 31795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:30.829934 31798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:30.821268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48617 (local address 127.0.0.1:46581)
0809 03:29:30.821779 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:30.821798 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:30.821841 (+ 43us) server_negotiation.cc:407] Connection header received
0809 03:29:30.822747 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:30.822776 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:30.823136 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:30.823493 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:30.823904 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:30.825352 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:30.826231 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:30.826922 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:30.827123 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:30.827880 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:30.827886 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:30.827901 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:30.828402 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:30.828470 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:30.828500 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:30.828647 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:30.828768 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 03:29:30.828980 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":144,"threads_started":1}
I20260809 03:29:30.830802 31795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54151954d35b4c9d843cd8f566829fd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } }
I20260809 03:29:30.831144 31795 leader_election.cc:302] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54151954d35b4c9d843cd8f566829fd4; no voters:
I20260809 03:29:30.831506 31795 leader_election.cc:288] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:30.831547 31800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:30.832086 31800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [term 1 LEADER]: Becoming Leader. State: Replica: 54151954d35b4c9d843cd8f566829fd4, State: Running, Role: LEADER
I20260809 03:29:30.832464 31800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54151954d35b4c9d843cd8f566829fd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } }
I20260809 03:29:30.832844 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:30.832878 31795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:30.834187 31802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54151954d35b4c9d843cd8f566829fd4. Latest consensus state: current_term: 1 leader_uuid: "54151954d35b4c9d843cd8f566829fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54151954d35b4c9d843cd8f566829fd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } } }
I20260809 03:29:30.834343 31802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:30.834757 31801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54151954d35b4c9d843cd8f566829fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54151954d35b4c9d843cd8f566829fd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46581 } } }
I20260809 03:29:30.835403 31801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4 [sys.catalog]: This master's current role is: LEADER
W20260809 03:29:30.837990 31820 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:29:30.838079 31820 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:29:30.838220 31816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:30.839099 31816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:30.841682 31798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:30.835260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36091 (local address 127.0.0.1:46581)
0809 03:29:30.835392 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:30.835397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:30.836836 (+ 1439us) server_negotiation.cc:407] Connection header received
0809 03:29:30.836962 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:30.836965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:30.837019 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:30.837113 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:30.837415 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:30.838176 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:30.839298 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:30.839632 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:30.839706 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:30.841161 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:30.841165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:30.841168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:30.841182 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:30.841210 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:30.841223 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:30.841304 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:30.841477 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 03:29:30.841533 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:29:30.845904 31816 catalog_manager.cc:1347] Generated new cluster ID: 769cf473c45d495ebb179ebc33466621
I20260809 03:29:30.845973 31816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:30.862588 31816 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:30.863523 31816 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:30.870334 31816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54151954d35b4c9d843cd8f566829fd4: Generated new TSK 0
I20260809 03:29:30.871030 31816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:30.878901 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 31722
2026-08-09T03:29:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:30 dist-test-slave-65mx krb5kdc[31827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:30 dist-test-slave-65mx krb5kdc[31827](info): set up 2 sockets
Aug 09 03:29:30 dist-test-slave-65mx krb5kdc[31827](info): commencing operation
krb5kdc: starting...
W20260809 03:29:31.975080 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:32 dist-test-slave-65mx krb5kdc[31827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:32.108080 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35473
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:32.220222 31840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:32.220525 31840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:32.220633 31840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:32.224690 31840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:32.224757 31840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:32.224778 31840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:32.224797 31840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:32.224812 31840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:32.230270 31840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:32.231348 31840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:32.232352 31840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:32.241357 31845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:32.241433 31846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:32.241559 31840 server_base.cc:1034] running on GCE node
W20260809 03:29:32.241761 31848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:32.242337 31840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:32.243425 31840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:32.244654 31840 hybrid_clock.cc:648] HybridClock initialized: now 1786246172244614 us; error 48 us; skew 500 ppm
Aug 09 03:29:32 dist-test-slave-65mx krb5kdc[31827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246172, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:32.247453 31840 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:32.248404 31840 webserver.cc:460] Webserver started at http://127.0.0.1:37459/ using document root <none> and password file <none>
I20260809 03:29:32.249022 31840 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:32.249117 31840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:32.249356 31840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:32.251405 31840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "b6c137bf8e234322a9c263ef6cfaf5db"
format_stamp: "Formatted at 2026-08-09 03:29:32 on dist-test-slave-65mx"
server_key: "6d20c45d2e4a5e90591f1924ced5142d"
server_key_iv: "4d36edfcfcb756665c9f6456db889579"
server_key_version: "encryptionkey@0"
I20260809 03:29:32.251984 31840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "b6c137bf8e234322a9c263ef6cfaf5db"
format_stamp: "Formatted at 2026-08-09 03:29:32 on dist-test-slave-65mx"
server_key: "6d20c45d2e4a5e90591f1924ced5142d"
server_key_iv: "4d36edfcfcb756665c9f6456db889579"
server_key_version: "encryptionkey@0"
I20260809 03:29:32.255712 31840 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 03:29:32.258162 31855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:32.259348 31840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:29:32.259490 31840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "b6c137bf8e234322a9c263ef6cfaf5db"
format_stamp: "Formatted at 2026-08-09 03:29:32 on dist-test-slave-65mx"
server_key: "6d20c45d2e4a5e90591f1924ced5142d"
server_key_iv: "4d36edfcfcb756665c9f6456db889579"
server_key_version: "encryptionkey@0"
I20260809 03:29:32.259601 31840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:32.264101 31840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:32.267038 31840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:32.267231 31840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:32.274667 31840 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35473
I20260809 03:29:32.274693 31909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35473 every 8 connection(s)
I20260809 03:29:32.275637 31840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:32.278590 31910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:32.284345 31910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db: Bootstrap starting.
I20260809 03:29:32.285319 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 31840
I20260809 03:29:32.285460 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:32.285743 9403 external_mini_cluster.cc:1428] Setting key 470aee77046074ba7335330ee4ff3e07
I20260809 03:29:32.287389 31910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:32.288158 31910 log.cc:824] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:32.290382 31910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db: No bootstrap required, opened a new log
I20260809 03:29:32.290812 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:32.286834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35405 (local address 127.0.0.1:35473)
0809 03:29:32.287277 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:32.287292 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:32.287326 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:29:32.287989 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:32.288006 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:32.288274 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:32.288646 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:32.289477 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:32.289483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:32.289496 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:32.289508 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:32.289568 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:32.289594 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:32.289717 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:32.289791 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 03:29:32.289931 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":146,"threads_started":1}
I20260809 03:29:32.293628 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:32.294468 31910 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c137bf8e234322a9c263ef6cfaf5db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35473 } }
I20260809 03:29:32.294672 31910 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:32.294731 31910 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6c137bf8e234322a9c263ef6cfaf5db, State: Initialized, Role: FOLLOWER
I20260809 03:29:32.295207 31910 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c137bf8e234322a9c263ef6cfaf5db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35473 } }
I20260809 03:29:32.295332 31910 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:32.295415 31910 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:32.295528 31910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:32.296696 31910 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c137bf8e234322a9c263ef6cfaf5db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35473 } }
I20260809 03:29:32.297060 31910 leader_election.cc:302] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6c137bf8e234322a9c263ef6cfaf5db; no voters:
I20260809 03:29:32.297405 31910 leader_election.cc:288] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:32.297547 31921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:32.297919 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:32.295955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51201 (local address 127.0.0.1:35473)
0809 03:29:32.296084 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:32.296088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:32.296345 (+ 257us) server_negotiation.cc:407] Connection header received
0809 03:29:32.296425 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:32.296428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:32.296462 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:32.296582 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:32.297389 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:32.297393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:32.297395 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:32.297407 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:32.297435 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:32.297449 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:32.297537 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:32.297732 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 03:29:32.297773 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:29:32.297798 31921 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [term 1 LEADER]: Becoming Leader. State: Replica: b6c137bf8e234322a9c263ef6cfaf5db, State: Running, Role: LEADER
I20260809 03:29:32.298604 31921 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c137bf8e234322a9c263ef6cfaf5db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35473 } }
I20260809 03:29:32.298722 31910 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:32.299862 31921 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6c137bf8e234322a9c263ef6cfaf5db. Latest consensus state: current_term: 1 leader_uuid: "b6c137bf8e234322a9c263ef6cfaf5db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c137bf8e234322a9c263ef6cfaf5db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35473 } } }
I20260809 03:29:32.300005 31921 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:32.300323 31922 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6c137bf8e234322a9c263ef6cfaf5db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c137bf8e234322a9c263ef6cfaf5db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35473 } } }
I20260809 03:29:32.300449 31922 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:32.300894 31930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:32.303864 31930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:32.309598 31930 catalog_manager.cc:1347] Generated new cluster ID: bc6274019f114343b92e6984e6552f04
I20260809 03:29:32.309680 31930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:32.320160 31930 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:32.320492 31930 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:32.331318 31930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6c137bf8e234322a9c263ef6cfaf5db: Generated new TSK 0
I20260809 03:29:32.332129 31930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:32.346258 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 31840
2026-08-09T03:29:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1468 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:32 dist-test-slave-65mx krb5kdc[31946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:32 dist-test-slave-65mx krb5kdc[31946](info): set up 2 sockets
Aug 09 03:29:32 dist-test-slave-65mx krb5kdc[31946](info): commencing operation
krb5kdc: starting...
W20260809 03:29:33.529111 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:33 dist-test-slave-65mx krb5kdc[31946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:33.660032 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:33.771592 31959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:33.771864 31959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:33.771906 31959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:33.775878 31959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:33.775943 31959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:33.775964 31959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:33.775980 31959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:33.775995 31959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:33.781241 31959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:33.782276 31959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:33.783226 31959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:33.791904 31965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:33.791883 31967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:33.792119 31959 server_base.cc:1034] running on GCE node
W20260809 03:29:33.792302 31964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:33.792968 31959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:33.794019 31959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:33.795197 31959 hybrid_clock.cc:648] HybridClock initialized: now 1786246173795176 us; error 31 us; skew 500 ppm
Aug 09 03:29:33 dist-test-slave-65mx krb5kdc[31946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246173, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:33.802491 31959 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:33.803534 31959 webserver.cc:460] Webserver started at http://127.0.0.1:33953/ using document root <none> and password file <none>
I20260809 03:29:33.804164 31959 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:33.804231 31959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:33.804663 31959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:33.806799 31959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "3613458f770f4a80b28eebb13dc63208"
format_stamp: "Formatted at 2026-08-09 03:29:33 on dist-test-slave-65mx"
server_key: "7b5527cbd8cd2eaaa2416a256e22d5ab"
server_key_iv: "eebe9468c1b7721b220c5d200f1da2ee"
server_key_version: "encryptionkey@0"
I20260809 03:29:33.807410 31959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "3613458f770f4a80b28eebb13dc63208"
format_stamp: "Formatted at 2026-08-09 03:29:33 on dist-test-slave-65mx"
server_key: "7b5527cbd8cd2eaaa2416a256e22d5ab"
server_key_iv: "eebe9468c1b7721b220c5d200f1da2ee"
server_key_version: "encryptionkey@0"
I20260809 03:29:33.811440 31959 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:29:33.814146 31974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:33.815443 31959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:29:33.815613 31959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "3613458f770f4a80b28eebb13dc63208"
format_stamp: "Formatted at 2026-08-09 03:29:33 on dist-test-slave-65mx"
server_key: "7b5527cbd8cd2eaaa2416a256e22d5ab"
server_key_iv: "eebe9468c1b7721b220c5d200f1da2ee"
server_key_version: "encryptionkey@0"
I20260809 03:29:33.815733 31959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:33.830415 31959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:33.833335 31959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:33.833560 31959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:33.841015 31959 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36251
I20260809 03:29:33.841032 32029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36251 every 8 connection(s)
I20260809 03:29:33.842036 31959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:33.844878 32030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:33.847846 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 31959
I20260809 03:29:33.847995 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:33.848318 9403 external_mini_cluster.cc:1428] Setting key 517f0de1f2e70480886b400f4408ff81
I20260809 03:29:33.851292 32030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208: Bootstrap starting.
I20260809 03:29:33.854050 32030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:33.854815 32030 log.cc:824] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:33.856932 32030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208: No bootstrap required, opened a new log
Aug 09 03:29:33 dist-test-slave-65mx krb5kdc[31946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:29:33.860895 32030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3613458f770f4a80b28eebb13dc63208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } }
I20260809 03:29:33.861155 32030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:33.861238 32030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3613458f770f4a80b28eebb13dc63208, State: Initialized, Role: FOLLOWER
I20260809 03:29:33.861687 32030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3613458f770f4a80b28eebb13dc63208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } }
I20260809 03:29:33.861842 32030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:33.861930 32030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:33.862053 32030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:33.862987 32030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3613458f770f4a80b28eebb13dc63208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } }
I20260809 03:29:33.863336 32030 leader_election.cc:302] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3613458f770f4a80b28eebb13dc63208; no voters:
I20260809 03:29:33.863699 32030 leader_election.cc:288] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:33.863819 32035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:33.864259 32035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [term 1 LEADER]: Becoming Leader. State: Replica: 3613458f770f4a80b28eebb13dc63208, State: Running, Role: LEADER
I20260809 03:29:33.864586 32035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3613458f770f4a80b28eebb13dc63208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } }
I20260809 03:29:33.864923 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:33.849759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:41357 (local address 127.0.0.1:36251)
0809 03:29:33.850174 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:33.850198 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:33.850234 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:29:33.851156 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:33.851183 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:33.851671 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:33.852052 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:33.853357 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:33.854527 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:33.855489 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:33.855996 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:33.856188 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:33.858960 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:33.858966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:33.858981 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:33.859009 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:33.861079 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:33.861841 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:33.861848 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:33.861855 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:33.862031 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:33.862435 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:33.862439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:33.862441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:33.862882 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:33.863196 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:33.863533 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 03:29:33.863806 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":162,"threads_started":1}
I20260809 03:29:33.865085 32030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:33.865795 32035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3613458f770f4a80b28eebb13dc63208. Latest consensus state: current_term: 1 leader_uuid: "3613458f770f4a80b28eebb13dc63208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3613458f770f4a80b28eebb13dc63208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } } }
I20260809 03:29:33.865936 32035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:33.866652 32043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:33.866626 32036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3613458f770f4a80b28eebb13dc63208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3613458f770f4a80b28eebb13dc63208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } } }
I20260809 03:29:33.866725 32036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:33.868829 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:33.870275 32043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:33.877393 32043 catalog_manager.cc:1347] Generated new cluster ID: 0ef5db98b1ae4099a50de15db08e156f
I20260809 03:29:33.877472 32043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:33.882567 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:33.872915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:38843 (local address 127.0.0.1:36251)
0809 03:29:33.873065 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:33.873069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:33.873426 (+ 357us) server_negotiation.cc:407] Connection header received
0809 03:29:33.873600 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:33.873604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:33.873668 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:33.873764 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:33.874509 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:33.875651 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:33.876684 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:33.876980 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:33.877184 (+ 204us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:33.878884 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:33.878888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:33.878891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:33.878918 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:33.880642 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:33.881184 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:33.881187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:33.881189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:33.881282 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:33.881638 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:33.881642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:33.881645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:33.881881 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:33.882023 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:33.882299 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 03:29:33.882405 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 03:29:33.897588 32043 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:33.898646 32043 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:33.911972 32043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3613458f770f4a80b28eebb13dc63208: Generated new TSK 0
I20260809 03:29:33.912801 32043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:33.922669 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 31959
W20260809 03:29:33.930805 31944 connection.cc:535] client connection to 127.0.0.1:36251 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36251): Connection reset by peer (error 104)
2026-08-09T03:29:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1576 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:33 dist-test-slave-65mx krb5kdc[32063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:33 dist-test-slave-65mx krb5kdc[32063](info): set up 2 sockets
Aug 09 03:29:33 dist-test-slave-65mx krb5kdc[32063](info): commencing operation
krb5kdc: starting...
W20260809 03:29:35.020793 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:35 dist-test-slave-65mx krb5kdc[32063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:35.147465 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36603
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:35.256207 32077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:35.256457 32077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:35.256498 32077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:35.260429 32077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:35.260484 32077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:35.260504 32077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:35.260520 32077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:35.260535 32077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:35.265704 32077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:35.266639 32077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:35.267624 32077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:35.276304 32082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:35.276331 32085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:35.276345 32083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:35.276479 32077 server_base.cc:1034] running on GCE node
I20260809 03:29:35.277227 32077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:35.278313 32077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:35.279506 32077 hybrid_clock.cc:648] HybridClock initialized: now 1786246175279496 us; error 30 us; skew 500 ppm
Aug 09 03:29:35 dist-test-slave-65mx krb5kdc[32063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:35.282311 32077 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:35.283425 32077 webserver.cc:460] Webserver started at http://127.0.0.1:38089/ using document root <none> and password file <none>
I20260809 03:29:35.284116 32077 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:35.284190 32077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:35.284406 32077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:35.286479 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "b185ec18558a432c879d1b9f7c82d7be"
format_stamp: "Formatted at 2026-08-09 03:29:35 on dist-test-slave-65mx"
server_key: "b351a9f0b1f2a5321a796f4dae3db726"
server_key_iv: "430c80f7f597b010fb904b3b4146db84"
server_key_version: "encryptionkey@0"
I20260809 03:29:35.287045 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "b185ec18558a432c879d1b9f7c82d7be"
format_stamp: "Formatted at 2026-08-09 03:29:35 on dist-test-slave-65mx"
server_key: "b351a9f0b1f2a5321a796f4dae3db726"
server_key_iv: "430c80f7f597b010fb904b3b4146db84"
server_key_version: "encryptionkey@0"
I20260809 03:29:35.290743 32077 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:29:35.293216 32092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:35.294469 32077 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:29:35.294597 32077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "b185ec18558a432c879d1b9f7c82d7be"
format_stamp: "Formatted at 2026-08-09 03:29:35 on dist-test-slave-65mx"
server_key: "b351a9f0b1f2a5321a796f4dae3db726"
server_key_iv: "430c80f7f597b010fb904b3b4146db84"
server_key_version: "encryptionkey@0"
I20260809 03:29:35.294704 32077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:35.326663 32077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:35.329699 32077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:35.329911 32077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:35.337575 32077 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36603
I20260809 03:29:35.337604 32147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36603 every 8 connection(s)
I20260809 03:29:35.338513 32077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:35.341460 32148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:35.344381 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 32077
I20260809 03:29:35.344493 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:35.344786 9403 external_mini_cluster.cc:1428] Setting key 997b83da9bd88f183053456784179d0c
I20260809 03:29:35.347784 32148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be: Bootstrap starting.
I20260809 03:29:35.350387 32148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:35.351243 32148 log.cc:824] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:35.353586 32148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be: No bootstrap required, opened a new log
I20260809 03:29:35.353840 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:35.346186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:33977 (local address 127.0.0.1:36603)
0809 03:29:35.346620 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:35.346635 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:35.346671 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:29:35.347296 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:35.347312 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:35.347637 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:35.347975 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:35.348388 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:35.349421 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:35.350338 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:35.350863 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:35.351033 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:35.352033 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:35.352041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:35.352056 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:35.352446 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:35.352498 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:35.352520 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:35.352696 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:35.352907 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 03:29:35.353053 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":137,"threads_started":1}
I20260809 03:29:35.356693 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:35.358028 32148 raft_consensus.cc:348] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185ec18558a432c879d1b9f7c82d7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36603 } }
I20260809 03:29:35.358361 32148 raft_consensus.cc:374] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:35.358434 32148 raft_consensus.cc:729] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b185ec18558a432c879d1b9f7c82d7be, State: Initialized, Role: FOLLOWER
I20260809 03:29:35.358919 32148 consensus_queue.cc:260] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185ec18558a432c879d1b9f7c82d7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36603 } }
I20260809 03:29:35.359103 32148 raft_consensus.cc:388] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:35.359192 32148 raft_consensus.cc:482] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:35.359305 32148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:35.360621 32148 raft_consensus.cc:504] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185ec18558a432c879d1b9f7c82d7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36603 } }
I20260809 03:29:35.360998 32148 leader_election.cc:302] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b185ec18558a432c879d1b9f7c82d7be; no voters:
I20260809 03:29:35.361392 32148 leader_election.cc:288] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:35.361503 32158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:35.361712 32158 raft_consensus.cc:686] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [term 1 LEADER]: Becoming Leader. State: Replica: b185ec18558a432c879d1b9f7c82d7be, State: Running, Role: LEADER
I20260809 03:29:35.362020 32158 consensus_queue.cc:237] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185ec18558a432c879d1b9f7c82d7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36603 } }
I20260809 03:29:35.362727 32148 sys_catalog.cc:564] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:35.363389 32160 sys_catalog.cc:455] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [sys.catalog]: SysCatalogTable state changed. Reason: New leader b185ec18558a432c879d1b9f7c82d7be. Latest consensus state: current_term: 1 leader_uuid: "b185ec18558a432c879d1b9f7c82d7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185ec18558a432c879d1b9f7c82d7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36603 } } }
I20260809 03:29:35.363538 32160 sys_catalog.cc:458] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:35.363989 32159 sys_catalog.cc:455] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b185ec18558a432c879d1b9f7c82d7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185ec18558a432c879d1b9f7c82d7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36603 } } }
I20260809 03:29:35.364184 32159 sys_catalog.cc:458] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:35.364362 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:35.359223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:53713 (local address 127.0.0.1:36603)
0809 03:29:35.359356 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:35.359362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:35.359571 (+ 209us) server_negotiation.cc:407] Connection header received
0809 03:29:35.359644 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:35.359647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:35.359696 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:35.359790 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:35.360021 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:35.360920 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:35.361984 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:35.362322 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:35.362398 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:35.363554 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:35.363559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:35.363562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:35.363577 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:35.363604 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:35.363618 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:35.363731 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:35.364114 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 03:29:35.364152 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 03:29:35.364814 32164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:35.368063 32164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:35.373478 32164 catalog_manager.cc:1347] Generated new cluster ID: 1cca494be4104b30a67ddba361b11abe
I20260809 03:29:35.373554 32164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:35.382383 32164 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:35.383409 32164 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:35.390439 32164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b185ec18558a432c879d1b9f7c82d7be: Generated new TSK 0
I20260809 03:29:35.391201 32164 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:35.401549 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 32077
W20260809 03:29:35.414786 32061 connection.cc:535] client connection to 127.0.0.1:36603 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36603): Connection reset by peer (error 104)
2026-08-09T03:29:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1484 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:35 dist-test-slave-65mx krb5kdc[32179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:35 dist-test-slave-65mx krb5kdc[32179](info): set up 2 sockets
Aug 09 03:29:35 dist-test-slave-65mx krb5kdc[32179](info): commencing operation
krb5kdc: starting...
W20260809 03:29:36.508908 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:36 dist-test-slave-65mx krb5kdc[32179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:36.641994 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:36.753170 32192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:36.753479 32192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:36.753522 32192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:36.757581 32192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:36.757647 32192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:36.757668 32192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:36.757685 32192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:36.757700 32192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:36.763185 32192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:36.764288 32192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:36.765354 32192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:36.773571 32200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:36.773573 32198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:36.773584 32202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:36.774006 32192 server_base.cc:1034] running on GCE node
I20260809 03:29:36.774577 32192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:36.775637 32192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:36.776826 32192 hybrid_clock.cc:648] HybridClock initialized: now 1786246176776814 us; error 31 us; skew 500 ppm
Aug 09 03:29:36 dist-test-slave-65mx krb5kdc[32179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:36.779734 32192 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:36.780624 32192 webserver.cc:460] Webserver started at http://127.0.0.1:38483/ using document root <none> and password file <none>
I20260809 03:29:36.781147 32192 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:36.781205 32192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:36.781368 32192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:36.783365 32192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "9e0fca6c0483467e8c22b2e2fbe454ed"
format_stamp: "Formatted at 2026-08-09 03:29:36 on dist-test-slave-65mx"
server_key: "2d887632d7b30bb3fb4a958ed9cf1979"
server_key_iv: "53ed6387f5eecada57ef2b63d36da93c"
server_key_version: "encryptionkey@0"
I20260809 03:29:36.783896 32192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "9e0fca6c0483467e8c22b2e2fbe454ed"
format_stamp: "Formatted at 2026-08-09 03:29:36 on dist-test-slave-65mx"
server_key: "2d887632d7b30bb3fb4a958ed9cf1979"
server_key_iv: "53ed6387f5eecada57ef2b63d36da93c"
server_key_version: "encryptionkey@0"
I20260809 03:29:36.787544 32192 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 03:29:36.790050 32211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:36.791265 32192 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:29:36.791400 32192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "9e0fca6c0483467e8c22b2e2fbe454ed"
format_stamp: "Formatted at 2026-08-09 03:29:36 on dist-test-slave-65mx"
server_key: "2d887632d7b30bb3fb4a958ed9cf1979"
server_key_iv: "53ed6387f5eecada57ef2b63d36da93c"
server_key_version: "encryptionkey@0"
I20260809 03:29:36.791512 32192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:36.796281 32192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:36.799715 32192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:36.799930 32192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:36.807235 32192 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37533
I20260809 03:29:36.807330 32266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37533 every 8 connection(s)
I20260809 03:29:36.808238 32192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:36.808784 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 32192
I20260809 03:29:36.808962 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:36.809314 9403 external_mini_cluster.cc:1428] Setting key 07a25c18fd992199d160bfa4f3e53353
I20260809 03:29:36.811900 32267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:36.814709 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:36.810825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:38171 (local address 127.0.0.1:37533)
0809 03:29:36.811283 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:36.811298 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:36.811333 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:29:36.812006 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:36.812028 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:36.812325 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:36.812676 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:36.813532 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:36.813537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:36.813548 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:36.813559 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:36.813615 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:36.813634 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:36.813793 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:36.813865 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 03:29:36.814027 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":146,"threads_started":1}
I20260809 03:29:36.817874 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:36.818971 32267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed: Bootstrap starting.
I20260809 03:29:36.821728 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:36.820003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:60425 (local address 127.0.0.1:37533)
0809 03:29:36.820157 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:36.820164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:36.820280 (+ 116us) server_negotiation.cc:407] Connection header received
0809 03:29:36.820364 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:36.820367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:36.820410 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:36.820504 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:36.821304 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:36.821308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:36.821310 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:36.821347 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:36.821380 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:36.821396 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:36.821487 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:36.821559 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 03:29:36.821601 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 03:29:36.822018 32267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:36.822947 32267 log.cc:824] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:36.825026 32267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed: No bootstrap required, opened a new log
I20260809 03:29:36.829044 32267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0fca6c0483467e8c22b2e2fbe454ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } }
I20260809 03:29:36.829294 32267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:36.829331 32267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0fca6c0483467e8c22b2e2fbe454ed, State: Initialized, Role: FOLLOWER
I20260809 03:29:36.829695 32267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0fca6c0483467e8c22b2e2fbe454ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } }
I20260809 03:29:36.829811 32267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:36.829851 32267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:36.829916 32267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:36.830878 32267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0fca6c0483467e8c22b2e2fbe454ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } }
I20260809 03:29:36.831183 32267 leader_election.cc:302] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e0fca6c0483467e8c22b2e2fbe454ed; no voters:
I20260809 03:29:36.831444 32267 leader_election.cc:288] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:36.831629 32277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:36.831921 32277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [term 1 LEADER]: Becoming Leader. State: Replica: 9e0fca6c0483467e8c22b2e2fbe454ed, State: Running, Role: LEADER
I20260809 03:29:36.832237 32277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0fca6c0483467e8c22b2e2fbe454ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } }
I20260809 03:29:36.832811 32267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:36.834438 32279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e0fca6c0483467e8c22b2e2fbe454ed. Latest consensus state: current_term: 1 leader_uuid: "9e0fca6c0483467e8c22b2e2fbe454ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0fca6c0483467e8c22b2e2fbe454ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } } }
I20260809 03:29:36.834489 32278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e0fca6c0483467e8c22b2e2fbe454ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0fca6c0483467e8c22b2e2fbe454ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37533 } } }
I20260809 03:29:36.834697 32278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:36.834733 32279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:36.835215 32291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:36.838107 32291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:36.843475 32291 catalog_manager.cc:1347] Generated new cluster ID: 42858ca58688462ea3d63cb834aadbb4
I20260809 03:29:36.843541 32291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:36.854864 32291 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:36.855199 32291 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:36.863590 32291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e0fca6c0483467e8c22b2e2fbe454ed: Generated new TSK 0
I20260809 03:29:36.864446 32291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:36.870002 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 32192
2026-08-09T03:29:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:36 dist-test-slave-65mx krb5kdc[32298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:36 dist-test-slave-65mx krb5kdc[32298](info): set up 2 sockets
Aug 09 03:29:36 dist-test-slave-65mx krb5kdc[32298](info): commencing operation
krb5kdc: starting...
W20260809 03:29:37.945869 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:38 dist-test-slave-65mx krb5kdc[32298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:38.071491 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36179
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:38.181598 32311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:38.181910 32311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:38.181988 32311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:38.185957 32311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:38.186060 32311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:38.186100 32311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:38.186156 32311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:38.186195 32311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:38.191414 32311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36179
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:38.192528 32311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:38.193573 32311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:38.202585 32317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:38.202585 32318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:38.202656 32320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:38.203058 32311 server_base.cc:1034] running on GCE node
I20260809 03:29:38.203557 32311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:38.204686 32311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:38.205852 32311 hybrid_clock.cc:648] HybridClock initialized: now 1786246178205825 us; error 44 us; skew 500 ppm
Aug 09 03:29:38 dist-test-slave-65mx krb5kdc[32298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246178, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:38.208737 32311 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:38.209668 32311 webserver.cc:460] Webserver started at http://127.0.0.1:46003/ using document root <none> and password file <none>
I20260809 03:29:38.210216 32311 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:38.210278 32311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:38.210493 32311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:38.212517 32311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "b787e05e1aa04e4a9a68a509130ba917"
format_stamp: "Formatted at 2026-08-09 03:29:38 on dist-test-slave-65mx"
server_key: "7247b93b965e6c17985f476a8f301a8f"
server_key_iv: "3954e4d888f428af3afcb790edad803d"
server_key_version: "encryptionkey@0"
I20260809 03:29:38.213186 32311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "b787e05e1aa04e4a9a68a509130ba917"
format_stamp: "Formatted at 2026-08-09 03:29:38 on dist-test-slave-65mx"
server_key: "7247b93b965e6c17985f476a8f301a8f"
server_key_iv: "3954e4d888f428af3afcb790edad803d"
server_key_version: "encryptionkey@0"
I20260809 03:29:38.216943 32311 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:29:38.219372 32327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:38.220791 32311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:38.220970 32311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "b787e05e1aa04e4a9a68a509130ba917"
format_stamp: "Formatted at 2026-08-09 03:29:38 on dist-test-slave-65mx"
server_key: "7247b93b965e6c17985f476a8f301a8f"
server_key_iv: "3954e4d888f428af3afcb790edad803d"
server_key_version: "encryptionkey@0"
I20260809 03:29:38.221112 32311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:38.252238 32311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:38.255366 32311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:38.255582 32311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:38.263031 32311 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36179
I20260809 03:29:38.263057 32383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36179 every 8 connection(s)
I20260809 03:29:38.263985 32311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:38.266878 32384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:38.268457 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 32311
I20260809 03:29:38.268646 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:38.269034 9403 external_mini_cluster.cc:1428] Setting key 586d9311bc74463db2756d40a51a30a5
I20260809 03:29:38.273577 32384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917: Bootstrap starting.
I20260809 03:29:38.275921 32384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:38.276902 32384 log.cc:824] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:29:38 dist-test-slave-65mx krb5kdc[32298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 03:29:38.279366 32384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917: No bootstrap required, opened a new log
I20260809 03:29:38.283921 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:38.270547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:56767 (local address 127.0.0.1:36179)
0809 03:29:38.270997 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:38.271011 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:38.271046 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:29:38.271661 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:38.271677 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:38.272010 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:38.272343 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:38.273225 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:38.274340 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:38.275222 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:38.275703 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:38.275869 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:38.279034 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:38.279039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:38.279051 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:38.279083 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:38.280868 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:38.281497 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:38.281501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:38.281505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:38.281620 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:38.282006 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:38.282008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:38.282009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:38.282392 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:38.282570 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:38.282920 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 03:29:38.283197 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":147,"threads_started":1}
I20260809 03:29:38.283785 32384 raft_consensus.cc:348] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787e05e1aa04e4a9a68a509130ba917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36179 } }
I20260809 03:29:38.284044 32384 raft_consensus.cc:374] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:38.284133 32384 raft_consensus.cc:729] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b787e05e1aa04e4a9a68a509130ba917, State: Initialized, Role: FOLLOWER
I20260809 03:29:38.284696 32384 consensus_queue.cc:260] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787e05e1aa04e4a9a68a509130ba917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36179 } }
I20260809 03:29:38.284864 32384 raft_consensus.cc:388] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:38.284970 32384 raft_consensus.cc:482] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:38.285085 32384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:38.286115 32384 raft_consensus.cc:504] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787e05e1aa04e4a9a68a509130ba917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36179 } }
I20260809 03:29:38.286504 32384 leader_election.cc:302] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b787e05e1aa04e4a9a68a509130ba917; no voters:
I20260809 03:29:38.286895 32384 leader_election.cc:288] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:38.287009 32389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:38.287091 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:38.287256 32389 raft_consensus.cc:686] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [term 1 LEADER]: Becoming Leader. State: Replica: b787e05e1aa04e4a9a68a509130ba917, State: Running, Role: LEADER
I20260809 03:29:38.287544 32389 consensus_queue.cc:237] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787e05e1aa04e4a9a68a509130ba917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36179 } }
I20260809 03:29:38.288295 32384 sys_catalog.cc:564] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:38.289000 32391 sys_catalog.cc:455] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b787e05e1aa04e4a9a68a509130ba917. Latest consensus state: current_term: 1 leader_uuid: "b787e05e1aa04e4a9a68a509130ba917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787e05e1aa04e4a9a68a509130ba917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36179 } } }
I20260809 03:29:38.289119 32391 sys_catalog.cc:458] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:38.289439 32390 sys_catalog.cc:455] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b787e05e1aa04e4a9a68a509130ba917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787e05e1aa04e4a9a68a509130ba917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36179 } } }
I20260809 03:29:38.289520 32390 sys_catalog.cc:458] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:38.289848 32399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:38.293699 32399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:38.300935 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:38.290467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:34469 (local address 127.0.0.1:36179)
0809 03:29:38.290580 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:38.290586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:38.291069 (+ 483us) server_negotiation.cc:407] Connection header received
0809 03:29:38.291174 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:38.291184 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:38.291251 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:38.291348 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:38.292226 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:38.293142 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:38.294735 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:38.295082 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:38.295193 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:38.297089 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:38.297093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:38.297096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:38.297127 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:38.299104 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:38.299607 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:38.299611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:38.299612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:38.299694 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:38.300030 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:38.300033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:38.300035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:38.300217 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:38.300373 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:38.300684 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 03:29:38.300783 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 03:29:38.300959 32399 catalog_manager.cc:1347] Generated new cluster ID: 761a6e00896f4096958db1d161afff87
I20260809 03:29:38.301460 32399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:38.330502 32399 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:38.331492 32399 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:38.335819 32399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b787e05e1aa04e4a9a68a509130ba917: Generated new TSK 0
I20260809 03:29:38.336705 32399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:38.345155 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 32311
W20260809 03:29:38.352520 32296 connection.cc:535] client connection to 127.0.0.1:36179 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36179): Connection reset by peer (error 104)
2026-08-09T03:29:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1474 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:38 dist-test-slave-65mx krb5kdc[32415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:38 dist-test-slave-65mx krb5kdc[32415](info): set up 2 sockets
Aug 09 03:29:38 dist-test-slave-65mx krb5kdc[32415](info): commencing operation
krb5kdc: starting...
W20260809 03:29:39.448906 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:39 dist-test-slave-65mx krb5kdc[32415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:39.580013 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33049
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:39.690800 32429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:39.691104 32429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:39.691202 32429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:39.695158 32429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:39.695252 32429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:39.695293 32429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:39.695350 32429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:39.695387 32429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:39.700608 32429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33049
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:39.701634 32429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:39.702649 32429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:39.711028 32435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:39.711208 32429 server_base.cc:1034] running on GCE node
W20260809 03:29:39.711112 32437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:39.711028 32434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:39.711971 32429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:39.713111 32429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:39.714309 32429 hybrid_clock.cc:648] HybridClock initialized: now 1786246179714292 us; error 30 us; skew 500 ppm
Aug 09 03:29:39 dist-test-slave-65mx krb5kdc[32415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246179, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:39.722198 32429 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:39.723088 32429 webserver.cc:460] Webserver started at http://127.0.0.1:37609/ using document root <none> and password file <none>
I20260809 03:29:39.723634 32429 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:39.723698 32429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:39.723865 32429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:39.726076 32429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "b38f8997ec794bcea69754c1ab45f947"
format_stamp: "Formatted at 2026-08-09 03:29:39 on dist-test-slave-65mx"
server_key: "e21b1c41c72108b2e0cd650a4ffc5fab"
server_key_iv: "e25f0defda420277aebe2d6bf930a366"
server_key_version: "encryptionkey@0"
I20260809 03:29:39.726701 32429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "b38f8997ec794bcea69754c1ab45f947"
format_stamp: "Formatted at 2026-08-09 03:29:39 on dist-test-slave-65mx"
server_key: "e21b1c41c72108b2e0cd650a4ffc5fab"
server_key_iv: "e25f0defda420277aebe2d6bf930a366"
server_key_version: "encryptionkey@0"
I20260809 03:29:39.730382 32429 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:29:39.732764 32444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:39.734037 32429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:39.734144 32429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "b38f8997ec794bcea69754c1ab45f947"
format_stamp: "Formatted at 2026-08-09 03:29:39 on dist-test-slave-65mx"
server_key: "e21b1c41c72108b2e0cd650a4ffc5fab"
server_key_iv: "e25f0defda420277aebe2d6bf930a366"
server_key_version: "encryptionkey@0"
I20260809 03:29:39.734246 32429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:39.759734 32429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:39.763139 32429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:39.763314 32429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:39.770727 32429 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33049
I20260809 03:29:39.770756 32500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33049 every 8 connection(s)
I20260809 03:29:39.771662 32429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:39.774441 32501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:39.777455 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 32429
I20260809 03:29:39.777572 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:39.777872 9403 external_mini_cluster.cc:1428] Setting key c831366bed0b2298cae74f2065d67581
I20260809 03:29:39.780350 32501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947: Bootstrap starting.
I20260809 03:29:39.782529 32501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:39.783288 32501 log.cc:824] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:39.785296 32501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947: No bootstrap required, opened a new log
I20260809 03:29:39.786731 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:39.779293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:49241 (local address 127.0.0.1:33049)
0809 03:29:39.779741 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:39.779756 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:39.779791 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:29:39.780395 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:39.780411 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:39.780845 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:39.781164 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:39.781571 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:39.782538 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:39.783452 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:39.783912 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:39.784083 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:39.785036 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:39.785041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:39.785053 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:39.785465 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:39.785530 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:39.785554 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:39.785698 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:39.785901 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 03:29:39.786042 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":146,"threads_started":1}
I20260809 03:29:39.789393 32501 raft_consensus.cc:348] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38f8997ec794bcea69754c1ab45f947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33049 } }
I20260809 03:29:39.789697 32501 raft_consensus.cc:374] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:39.789734 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:39.789780 32501 raft_consensus.cc:729] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38f8997ec794bcea69754c1ab45f947, State: Initialized, Role: FOLLOWER
I20260809 03:29:39.790225 32501 consensus_queue.cc:260] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38f8997ec794bcea69754c1ab45f947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33049 } }
I20260809 03:29:39.790378 32501 raft_consensus.cc:388] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:39.790449 32501 raft_consensus.cc:482] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:39.790548 32501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:39.791544 32501 raft_consensus.cc:504] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38f8997ec794bcea69754c1ab45f947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33049 } }
I20260809 03:29:39.791929 32501 leader_election.cc:302] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b38f8997ec794bcea69754c1ab45f947; no voters:
I20260809 03:29:39.792384 32501 leader_election.cc:288] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:39.792510 32512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:39.792769 32512 raft_consensus.cc:686] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [term 1 LEADER]: Becoming Leader. State: Replica: b38f8997ec794bcea69754c1ab45f947, State: Running, Role: LEADER
I20260809 03:29:39.793187 32512 consensus_queue.cc:237] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38f8997ec794bcea69754c1ab45f947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33049 } }
I20260809 03:29:39.793720 32501 sys_catalog.cc:564] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:39.794407 32512 sys_catalog.cc:455] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b38f8997ec794bcea69754c1ab45f947. Latest consensus state: current_term: 1 leader_uuid: "b38f8997ec794bcea69754c1ab45f947" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38f8997ec794bcea69754c1ab45f947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33049 } } }
I20260809 03:29:39.795132 32512 sys_catalog.cc:458] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:39.795017 32513 sys_catalog.cc:455] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b38f8997ec794bcea69754c1ab45f947" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38f8997ec794bcea69754c1ab45f947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33049 } } }
I20260809 03:29:39.795495 32513 sys_catalog.cc:458] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:39.795846 32524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:39.796975 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:39.791907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:36715 (local address 127.0.0.1:33049)
0809 03:29:39.792039 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:39.792046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:39.792211 (+ 165us) server_negotiation.cc:407] Connection header received
0809 03:29:39.792292 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:39.792298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:39.792357 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:39.792450 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:39.792830 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:39.793568 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:39.794752 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:39.795141 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:39.795228 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:39.796396 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:39.796399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:39.796403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:39.796416 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:39.796448 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:39.796465 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:39.796612 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:39.796781 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 03:29:39.796818 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:29:39.799675 32524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:39.805105 32524 catalog_manager.cc:1347] Generated new cluster ID: 421a0cead4ac43e99b9372a6a709cce6
I20260809 03:29:39.805191 32524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:39.842352 32524 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:39.843365 32524 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:39.847393 32524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b38f8997ec794bcea69754c1ab45f947: Generated new TSK 0
I20260809 03:29:39.848157 32524 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:39.852687 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 32429
W20260809 03:29:39.860103 32413 connection.cc:535] client connection to 127.0.0.1:33049 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33049): Connection reset by peer (error 104)
2026-08-09T03:29:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:39 dist-test-slave-65mx krb5kdc[32535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:39 dist-test-slave-65mx krb5kdc[32535](info): set up 2 sockets
Aug 09 03:29:39 dist-test-slave-65mx krb5kdc[32535](info): commencing operation
krb5kdc: starting...
W20260809 03:29:40.946988 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:41 dist-test-slave-65mx krb5kdc[32535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 03:29:41.083194 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36393
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:41.197445 32549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:41.197763 32549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:41.197842 32549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:41.201946 32549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:41.202049 32549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:41.202088 32549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:41.202147 32549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:41.202186 32549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:41.207542 32549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36393
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:41.208748 32549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:41.209800 32549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:41.218446 32554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:41.218560 32557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:41.218457 32555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:41.218703 32549 server_base.cc:1034] running on GCE node
I20260809 03:29:41.219334 32549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:41.220466 32549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:41.221717 32549 hybrid_clock.cc:648] HybridClock initialized: now 1786246181221701 us; error 39 us; skew 500 ppm
Aug 09 03:29:41 dist-test-slave-65mx krb5kdc[32535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246181, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:41.224781 32549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 03:29:41.225749 32549 webserver.cc:460] Webserver started at http://127.0.0.1:33789/ using document root <none> and password file <none>
I20260809 03:29:41.226291 32549 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:41.226352 32549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:41.226529 32549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:41.228686 32549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "1a6dcba629f64bda827e470eab30ff57"
format_stamp: "Formatted at 2026-08-09 03:29:41 on dist-test-slave-65mx"
server_key: "c2964b788fdfcb218c447dfe8c053475"
server_key_iv: "264b33d08062d5eb703b8bd67c857f31"
server_key_version: "encryptionkey@0"
I20260809 03:29:41.229292 32549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "1a6dcba629f64bda827e470eab30ff57"
format_stamp: "Formatted at 2026-08-09 03:29:41 on dist-test-slave-65mx"
server_key: "c2964b788fdfcb218c447dfe8c053475"
server_key_iv: "264b33d08062d5eb703b8bd67c857f31"
server_key_version: "encryptionkey@0"
I20260809 03:29:41.233248 32549 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 03:29:41.235682 32564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:41.237102 32549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 03:29:41.237238 32549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "1a6dcba629f64bda827e470eab30ff57"
format_stamp: "Formatted at 2026-08-09 03:29:41 on dist-test-slave-65mx"
server_key: "c2964b788fdfcb218c447dfe8c053475"
server_key_iv: "264b33d08062d5eb703b8bd67c857f31"
server_key_version: "encryptionkey@0"
I20260809 03:29:41.237347 32549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:41.242110 32549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:41.245428 32549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:41.245635 32549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:41.253005 32549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36393
I20260809 03:29:41.253031 32618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36393 every 8 connection(s)
I20260809 03:29:41.253944 32549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:41.256682 32619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:41.260422 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 32549
I20260809 03:29:41.260605 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:41.260931 9403 external_mini_cluster.cc:1428] Setting key e8bc6152a5f5e10ba66e57d4a62f1e5f
I20260809 03:29:41.262583 32619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57: Bootstrap starting.
I20260809 03:29:41.264858 32619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:41.265806 32619 log.cc:824] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:41.266451 32622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:41.262340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:56863 (local address 127.0.0.1:36393)
0809 03:29:41.262820 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:41.262834 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:41.262869 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:29:41.263521 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:41.263537 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:41.263803 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:41.264144 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:41.265066 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:41.265075 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:41.265091 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:41.265110 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:41.265188 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:41.265218 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:41.265330 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:41.265464 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 03:29:41.265618 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":137,"threads_started":1}
I20260809 03:29:41.267966 32619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57: No bootstrap required, opened a new log
I20260809 03:29:41.269477 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:41.272046 32619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6dcba629f64bda827e470eab30ff57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36393 } }
I20260809 03:29:41.272274 32619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:41.272351 32619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a6dcba629f64bda827e470eab30ff57, State: Initialized, Role: FOLLOWER
I20260809 03:29:41.272811 32619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6dcba629f64bda827e470eab30ff57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36393 } }
I20260809 03:29:41.272986 32619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:41.273090 32619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:41.273214 32619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:41.274266 32622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:41.271640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:49397 (local address 127.0.0.1:36393)
0809 03:29:41.271749 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:41.271753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:41.272875 (+ 1122us) server_negotiation.cc:407] Connection header received
0809 03:29:41.272971 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:41.272974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:41.273015 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:41.273104 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:41.273877 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:41.273881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:41.273883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:41.273895 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:41.273925 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:41.273937 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:41.274020 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:41.274104 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 03:29:41.274148 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:29:41.274477 32619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6dcba629f64bda827e470eab30ff57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36393 } }
I20260809 03:29:41.274832 32619 leader_election.cc:302] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a6dcba629f64bda827e470eab30ff57; no voters:
I20260809 03:29:41.275168 32619 leader_election.cc:288] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:41.275249 32629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:41.275467 32629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [term 1 LEADER]: Becoming Leader. State: Replica: 1a6dcba629f64bda827e470eab30ff57, State: Running, Role: LEADER
I20260809 03:29:41.275828 32629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6dcba629f64bda827e470eab30ff57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36393 } }
I20260809 03:29:41.276497 32619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:41.277715 32630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a6dcba629f64bda827e470eab30ff57. Latest consensus state: current_term: 1 leader_uuid: "1a6dcba629f64bda827e470eab30ff57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6dcba629f64bda827e470eab30ff57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36393 } } }
I20260809 03:29:41.277855 32630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:41.278149 32631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a6dcba629f64bda827e470eab30ff57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a6dcba629f64bda827e470eab30ff57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36393 } } }
I20260809 03:29:41.278326 32631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:41.278247 32639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:41.281694 32639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:41.286759 32639 catalog_manager.cc:1347] Generated new cluster ID: 352c08ee65214506ad55b3df8dc32293
I20260809 03:29:41.286832 32639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:41.305311 32639 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:41.305635 32639 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:41.317778 32639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a6dcba629f64bda827e470eab30ff57: Generated new TSK 0
I20260809 03:29:41.318444 32639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:41.322477 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 32549
2026-08-09T03:29:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1469 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:41 dist-test-slave-65mx krb5kdc[32653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:41 dist-test-slave-65mx krb5kdc[32653](info): set up 2 sockets
Aug 09 03:29:41 dist-test-slave-65mx krb5kdc[32653](info): commencing operation
krb5kdc: starting...
W20260809 03:29:42.410449 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:42 dist-test-slave-65mx krb5kdc[32653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:42.544752 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:36535
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:37629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:36535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:42.655961 32666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:42.656224 32666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:42.656265 32666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:42.660316 32666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:42.660382 32666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:42.660401 32666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:42.660419 32666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:42.660434 32666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:42.666343 32666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:37629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:36535
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:36535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:42.667407 32666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:42.668426 32666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:42.676697 32673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:42.676710 32676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:42.676714 32674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:42.677001 32666 server_base.cc:1034] running on GCE node
I20260809 03:29:42.677709 32666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:42.679116 32666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:42.680289 32666 hybrid_clock.cc:648] HybridClock initialized: now 1786246182680268 us; error 44 us; skew 500 ppm
Aug 09 03:29:42 dist-test-slave-65mx krb5kdc[32653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:42.683171 32666 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:42.684027 32666 webserver.cc:460] Webserver started at http://127.9.46.254:41117/ using document root <none> and password file <none>
I20260809 03:29:42.684629 32666 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:42.684721 32666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:42.684944 32666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:42.686981 32666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "df48d92b9e8944daa0b558b5c37367ea"
format_stamp: "Formatted at 2026-08-09 03:29:42 on dist-test-slave-65mx"
server_key: "31ff65f4ca5cd45c204de9b8db17c057"
server_key_iv: "54c76dca8bc11fb73799fb9ffebdd5ed"
server_key_version: "encryptionkey@0"
I20260809 03:29:42.687533 32666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "df48d92b9e8944daa0b558b5c37367ea"
format_stamp: "Formatted at 2026-08-09 03:29:42 on dist-test-slave-65mx"
server_key: "31ff65f4ca5cd45c204de9b8db17c057"
server_key_iv: "54c76dca8bc11fb73799fb9ffebdd5ed"
server_key_version: "encryptionkey@0"
I20260809 03:29:42.691133 32666 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 03:29:42.693463 32683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:42.694648 32666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:42.694792 32666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "df48d92b9e8944daa0b558b5c37367ea"
format_stamp: "Formatted at 2026-08-09 03:29:42 on dist-test-slave-65mx"
server_key: "31ff65f4ca5cd45c204de9b8db17c057"
server_key_iv: "54c76dca8bc11fb73799fb9ffebdd5ed"
server_key_version: "encryptionkey@0"
I20260809 03:29:42.694900 32666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:42.713640 32666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:42.716848 32666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:42.717069 32666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:42.724331 32666 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:36535
I20260809 03:29:42.724357 32737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:36535 every 8 connection(s)
I20260809 03:29:42.725288 32666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:42.728031 32738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:42.731603 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 32666
I20260809 03:29:42.731760 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:42.732085 9403 external_mini_cluster.cc:1428] Setting key 1bd54fdee076fe760a67c392f13dea7d
I20260809 03:29:42.734143 32738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea: Bootstrap starting.
I20260809 03:29:42.736299 32738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:42.737124 32738 log.cc:824] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:42.738924 32738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea: No bootstrap required, opened a new log
Aug 09 03:29:42 dist-test-slave-65mx krb5kdc[32653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:42.742926 32738 raft_consensus.cc:348] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df48d92b9e8944daa0b558b5c37367ea" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36535 } }
I20260809 03:29:42.743160 32738 raft_consensus.cc:374] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:42.743204 32738 raft_consensus.cc:729] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df48d92b9e8944daa0b558b5c37367ea, State: Initialized, Role: FOLLOWER
I20260809 03:29:42.743697 32738 consensus_queue.cc:260] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df48d92b9e8944daa0b558b5c37367ea" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36535 } }
I20260809 03:29:42.743826 32738 raft_consensus.cc:388] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:42.743911 32738 raft_consensus.cc:482] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:42.744029 32738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:42.745244 32738 raft_consensus.cc:504] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df48d92b9e8944daa0b558b5c37367ea" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36535 } }
I20260809 03:29:42.745601 32738 leader_election.cc:302] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df48d92b9e8944daa0b558b5c37367ea; no voters:
I20260809 03:29:42.745952 32738 leader_election.cc:288] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:42.746053 32743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:42.746338 32743 raft_consensus.cc:686] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [term 1 LEADER]: Becoming Leader. State: Replica: df48d92b9e8944daa0b558b5c37367ea, State: Running, Role: LEADER
I20260809 03:29:42.746695 32743 consensus_queue.cc:237] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df48d92b9e8944daa0b558b5c37367ea" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36535 } }
I20260809 03:29:42.747274 32738 sys_catalog.cc:564] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:42.747378 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:42.733600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39865 (local address 127.9.46.254:36535)
0809 03:29:42.734034 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:42.734049 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:42.734086 (+ 37us) server_negotiation.cc:407] Connection header received
0809 03:29:42.734785 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:42.734803 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:42.735128 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:42.735457 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:42.736279 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:42.737446 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:42.738315 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:42.738773 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:42.738940 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:42.741856 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:42.741861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:42.741874 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:42.741901 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:42.743845 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:42.744495 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:42.744507 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:42.744512 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:42.744701 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:42.745055 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:42.745057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:42.745058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:42.745508 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:42.745664 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:42.746129 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 03:29:42.746403 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":128,"threads_started":1}
I20260809 03:29:42.747864 32745 sys_catalog.cc:455] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader df48d92b9e8944daa0b558b5c37367ea. Latest consensus state: current_term: 1 leader_uuid: "df48d92b9e8944daa0b558b5c37367ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df48d92b9e8944daa0b558b5c37367ea" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36535 } } }
I20260809 03:29:42.748001 32745 sys_catalog.cc:458] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:42.748420 32750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:42.750377 32744 sys_catalog.cc:455] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df48d92b9e8944daa0b558b5c37367ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df48d92b9e8944daa0b558b5c37367ea" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 36535 } } }
I20260809 03:29:42.750545 32744 sys_catalog.cc:458] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:42.750890 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:42.752118 32750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:42.758788 32750 catalog_manager.cc:1347] Generated new cluster ID: 2fb49fcb334b4f3682846c2807eb6c90
I20260809 03:29:42.759133 32750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:42.762091 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:42.753191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36975 (local address 127.9.46.254:36535)
0809 03:29:42.753303 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:42.753308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:42.753500 (+ 192us) server_negotiation.cc:407] Connection header received
0809 03:29:42.753625 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:42.753628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:42.753672 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:42.753759 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:42.754413 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:42.755186 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:42.756285 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:42.756652 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:42.756714 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:42.758560 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:42.758563 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:42.758566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:42.758591 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:42.760186 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:42.760763 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:42.760766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:42.760768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:42.760861 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:42.761229 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:42.761232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:42.761234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:42.761407 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:42.761568 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:42.761841 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 03:29:42.761936 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 03:29:42.790334 32750 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:42.791597 32750 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:42.798942 32750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df48d92b9e8944daa0b558b5c37367ea: Generated new TSK 0
I20260809 03:29:42.799773 32750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:42.810920 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 32666
W20260809 03:29:42.818640 32651 connection.cc:535] client connection to 127.9.46.254:36535 recv error: Network error: failed to read from TLS socket (remote: 127.9.46.254:36535): Connection reset by peer (error 104)
2026-08-09T03:29:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:42 dist-test-slave-65mx krb5kdc[304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:42 dist-test-slave-65mx krb5kdc[304](info): set up 2 sockets
Aug 09 03:29:42 dist-test-slave-65mx krb5kdc[304](info): commencing operation
krb5kdc: starting...
W20260809 03:29:43.907298 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:43 dist-test-slave-65mx krb5kdc[304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:44.035660 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46619
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:42969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:46619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:44.160180 319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:44.160580 319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:44.160697 319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:44.164855 319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:44.164937 319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:44.164976 319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:44.165037 319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:44.165074 319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:44.170485 319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:42969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:46619
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:46619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:44.171595 319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:44.172679 319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:44.180954 328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:44.180950 326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:44.180969 325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:44.181111 319 server_base.cc:1034] running on GCE node
I20260809 03:29:44.181826 319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:44.182940 319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:44.184139 319 hybrid_clock.cc:648] HybridClock initialized: now 1786246184184129 us; error 60 us; skew 500 ppm
Aug 09 03:29:44 dist-test-slave-65mx krb5kdc[304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246184, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:44.187095 319 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:44.188050 319 webserver.cc:460] Webserver started at http://127.9.46.254:32947/ using document root <none> and password file <none>
I20260809 03:29:44.188655 319 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:44.188736 319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:44.188915 319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:44.190968 319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "9426a77232bb4f36986870586faedd45"
format_stamp: "Formatted at 2026-08-09 03:29:44 on dist-test-slave-65mx"
server_key: "03e086b348a8d87951a95f394f11a382"
server_key_iv: "fe523e172b4e63d6a89e7ddb20aa7c5d"
server_key_version: "encryptionkey@0"
I20260809 03:29:44.191504 319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "9426a77232bb4f36986870586faedd45"
format_stamp: "Formatted at 2026-08-09 03:29:44 on dist-test-slave-65mx"
server_key: "03e086b348a8d87951a95f394f11a382"
server_key_iv: "fe523e172b4e63d6a89e7ddb20aa7c5d"
server_key_version: "encryptionkey@0"
I20260809 03:29:44.195385 319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:29:44.197957 335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:44.199307 319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 03:29:44.199422 319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "9426a77232bb4f36986870586faedd45"
format_stamp: "Formatted at 2026-08-09 03:29:44 on dist-test-slave-65mx"
server_key: "03e086b348a8d87951a95f394f11a382"
server_key_iv: "fe523e172b4e63d6a89e7ddb20aa7c5d"
server_key_version: "encryptionkey@0"
I20260809 03:29:44.199587 319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:44.231359 319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:44.234910 319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:44.235206 319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:44.243176 319 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:46619
I20260809 03:29:44.243206 389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:46619 every 8 connection(s)
I20260809 03:29:44.244130 319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:44.246826 390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:44.252779 390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45: Bootstrap starting.
I20260809 03:29:44.253259 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 319
I20260809 03:29:44.253410 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:44.253710 9403 external_mini_cluster.cc:1428] Setting key 29caac996282f2537b837513653b89a8
I20260809 03:29:44.255566 390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:44.256278 390 log.cc:824] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:44.258185 390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45: No bootstrap required, opened a new log
I20260809 03:29:44.261857 390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426a77232bb4f36986870586faedd45" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46619 } }
I20260809 03:29:44.262161 390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:44.262384 390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9426a77232bb4f36986870586faedd45, State: Initialized, Role: FOLLOWER
I20260809 03:29:44.262861 390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426a77232bb4f36986870586faedd45" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46619 } }
I20260809 03:29:44.262915 393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:44.255034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47353 (local address 127.9.46.254:46619)
0809 03:29:44.255496 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:44.255510 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:44.255545 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:29:44.256235 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:44.256252 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:44.256690 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:44.257027 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:44.257458 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:44.258456 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:44.259342 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:44.259829 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:44.260046 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:44.261048 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:44.261053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:44.261065 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:44.261443 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:44.261497 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:44.261519 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:44.261677 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:44.261871 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 03:29:44.262013 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":146,"threads_started":1}
I20260809 03:29:44.263037 390 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:44.263118 390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:44.263223 390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:44.264356 390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426a77232bb4f36986870586faedd45" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46619 } }
I20260809 03:29:44.264752 390 leader_election.cc:302] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9426a77232bb4f36986870586faedd45; no voters:
I20260809 03:29:44.265153 390 leader_election.cc:288] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:44.265241 395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:44.265497 395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [term 1 LEADER]: Becoming Leader. State: Replica: 9426a77232bb4f36986870586faedd45, State: Running, Role: LEADER
I20260809 03:29:44.266150 395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426a77232bb4f36986870586faedd45" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46619 } }
I20260809 03:29:44.266315 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:44.266551 390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:44.267871 396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9426a77232bb4f36986870586faedd45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426a77232bb4f36986870586faedd45" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46619 } } }
I20260809 03:29:44.267954 397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9426a77232bb4f36986870586faedd45. Latest consensus state: current_term: 1 leader_uuid: "9426a77232bb4f36986870586faedd45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9426a77232bb4f36986870586faedd45" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 46619 } } }
I20260809 03:29:44.268014 396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:44.268038 397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45 [sys.catalog]: This master's current role is: LEADER
W20260809 03:29:44.271255 417 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:29:44.271353 417 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:29:44.271456 410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:44.272135 410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:44.274155 393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:44.269081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39035 (local address 127.9.46.254:46619)
0809 03:29:44.269185 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:44.269190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:44.269472 (+ 282us) server_negotiation.cc:407] Connection header received
0809 03:29:44.269660 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:44.269663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:44.269710 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:44.269800 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:44.270162 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:44.270927 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:44.272094 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:44.272420 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:44.272514 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:44.273635 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:44.273640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:44.273643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:44.273656 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:44.273685 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:44.273698 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:44.273789 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:44.273977 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 03:29:44.274013 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 03:29:44.278103 410 catalog_manager.cc:1347] Generated new cluster ID: 63870e16316b44bc90f44b40a0302ce5
I20260809 03:29:44.278190 410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:44.294589 410 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:44.295852 410 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:44.305394 410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9426a77232bb4f36986870586faedd45: Generated new TSK 0
I20260809 03:29:44.306102 410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:44.315424 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 319
W20260809 03:29:44.322433 302 connection.cc:535] client connection to 127.9.46.254:46619 recv error: Network error: failed to read from TLS socket (remote: 127.9.46.254:46619): Connection reset by peer (error 104)
2026-08-09T03:29:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:44 dist-test-slave-65mx krb5kdc[424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:44 dist-test-slave-65mx krb5kdc[424](info): set up 2 sockets
Aug 09 03:29:44 dist-test-slave-65mx krb5kdc[424](info): commencing operation
krb5kdc: starting...
W20260809 03:29:45.488204 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:45 dist-test-slave-65mx krb5kdc[424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:45.616278 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:38961
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:38961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:45.728616 437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:45.728888 437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:45.728931 437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:45.732911 437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:45.732975 437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:45.732995 437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:45.733011 437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:45.733027 437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:45.738294 437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:38961
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:38961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:45.739308 437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:45.740381 437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:45.748823 443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:45.748842 444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:45.748834 446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:45.749027 437 server_base.cc:1034] running on GCE node
I20260809 03:29:45.749688 437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:45.750849 437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:45.752063 437 hybrid_clock.cc:648] HybridClock initialized: now 1786246185752041 us; error 33 us; skew 500 ppm
Aug 09 03:29:45 dist-test-slave-65mx krb5kdc[424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246185, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:45.759733 437 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:45.760748 437 webserver.cc:460] Webserver started at http://127.9.46.254:35869/ using document root <none> and password file <none>
I20260809 03:29:45.761281 437 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:45.761340 437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:45.761513 437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:45.763549 437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "730743e6610c48cead6d751f5736cdc4"
format_stamp: "Formatted at 2026-08-09 03:29:45 on dist-test-slave-65mx"
server_key: "79bd7dade4c0423d03336f9364e3b3d4"
server_key_iv: "1f5c5a3c23dbbf54f97b7490c029a798"
server_key_version: "encryptionkey@0"
I20260809 03:29:45.764058 437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "730743e6610c48cead6d751f5736cdc4"
format_stamp: "Formatted at 2026-08-09 03:29:45 on dist-test-slave-65mx"
server_key: "79bd7dade4c0423d03336f9364e3b3d4"
server_key_iv: "1f5c5a3c23dbbf54f97b7490c029a798"
server_key_version: "encryptionkey@0"
I20260809 03:29:45.767740 437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 03:29:45.770258 454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:45.771690 437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 03:29:45.771814 437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "730743e6610c48cead6d751f5736cdc4"
format_stamp: "Formatted at 2026-08-09 03:29:45 on dist-test-slave-65mx"
server_key: "79bd7dade4c0423d03336f9364e3b3d4"
server_key_iv: "1f5c5a3c23dbbf54f97b7490c029a798"
server_key_version: "encryptionkey@0"
I20260809 03:29:45.771915 437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:45.776649 437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:45.779945 437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:45.780104 437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:45.787498 437 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:38961
I20260809 03:29:45.787514 508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:38961 every 8 connection(s)
I20260809 03:29:45.788441 437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:45.791301 510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:45.793198 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 437
I20260809 03:29:45.793347 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:45.793661 9403 external_mini_cluster.cc:1428] Setting key 53975787ceea6817291945b94ec999fe
I20260809 03:29:45.797610 510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4: Bootstrap starting.
I20260809 03:29:45.799468 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:45.795337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49921 (local address 127.9.46.254:38961)
0809 03:29:45.795786 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:45.795801 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:45.795837 (+ 36us) server_negotiation.cc:407] Connection header received
0809 03:29:45.796728 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:45.796747 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:45.797044 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:45.797381 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:45.798249 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:45.798255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:45.798275 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:45.798286 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:45.798355 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:45.798384 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:45.798535 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:45.798616 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 03:29:45.798775 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":125,"threads_started":1}
I20260809 03:29:45.800073 510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:45.800978 510 log.cc:824] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:45.802421 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:45.803150 510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4: No bootstrap required, opened a new log
I20260809 03:29:45.806380 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:45.804624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44933 (local address 127.9.46.254:38961)
0809 03:29:45.804753 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:45.804757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:45.804962 (+ 205us) server_negotiation.cc:407] Connection header received
0809 03:29:45.805098 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:45.805101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:45.805133 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:45.805198 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:45.805927 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:45.805929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:45.805931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:45.805942 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:45.805980 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:45.805991 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:45.806073 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:45.806201 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 03:29:45.806234 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 03:29:45.807809 510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730743e6610c48cead6d751f5736cdc4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38961 } }
I20260809 03:29:45.808076 510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:45.808148 510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730743e6610c48cead6d751f5736cdc4, State: Initialized, Role: FOLLOWER
I20260809 03:29:45.808598 510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730743e6610c48cead6d751f5736cdc4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38961 } }
I20260809 03:29:45.808759 510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:45.808827 510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:45.808924 510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:45.809900 510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730743e6610c48cead6d751f5736cdc4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38961 } }
I20260809 03:29:45.810241 510 leader_election.cc:302] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 730743e6610c48cead6d751f5736cdc4; no voters:
I20260809 03:29:45.810585 510 leader_election.cc:288] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:45.810720 520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:45.810967 520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [term 1 LEADER]: Becoming Leader. State: Replica: 730743e6610c48cead6d751f5736cdc4, State: Running, Role: LEADER
I20260809 03:29:45.811317 520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730743e6610c48cead6d751f5736cdc4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38961 } }
I20260809 03:29:45.811872 510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:45.813154 522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 730743e6610c48cead6d751f5736cdc4. Latest consensus state: current_term: 1 leader_uuid: "730743e6610c48cead6d751f5736cdc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730743e6610c48cead6d751f5736cdc4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38961 } } }
I20260809 03:29:45.813338 522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:45.813651 521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "730743e6610c48cead6d751f5736cdc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730743e6610c48cead6d751f5736cdc4" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 38961 } } }
I20260809 03:29:45.813827 521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:45.814422 530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:45.817015 530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:45.822032 530 catalog_manager.cc:1347] Generated new cluster ID: fb2b36c837774f69b23f93636a430ec9
I20260809 03:29:45.822103 530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:45.838668 530 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:45.839017 530 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:45.844689 530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 730743e6610c48cead6d751f5736cdc4: Generated new TSK 0
I20260809 03:29:45.845378 530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:45.850108 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 437
2026-08-09T03:29:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1534 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:45 dist-test-slave-65mx krb5kdc[543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:45 dist-test-slave-65mx krb5kdc[543](info): set up 2 sockets
Aug 09 03:29:45 dist-test-slave-65mx krb5kdc[543](info): commencing operation
krb5kdc: starting...
W20260809 03:29:46.946528 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:47 dist-test-slave-65mx krb5kdc[543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:47.084877 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39849
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:42559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:39849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:47.200775 557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:47.201062 557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:47.201113 557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:47.205133 557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:47.205202 557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:47.205222 557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:47.205240 557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:47.205255 557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:47.210566 557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:42559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:39849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:47.211676 557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:47.212711 557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:47.221601 563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:47.221607 566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:47.221683 564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:47.221879 557 server_base.cc:1034] running on GCE node
I20260809 03:29:47.222616 557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:47.223769 557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:47.224977 557 hybrid_clock.cc:648] HybridClock initialized: now 1786246187224953 us; error 34 us; skew 500 ppm
Aug 09 03:29:47 dist-test-slave-65mx krb5kdc[543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246187, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:47.227851 557 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:47.228885 557 webserver.cc:460] Webserver started at http://127.9.46.254:43341/ using document root <none> and password file <none>
I20260809 03:29:47.229472 557 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:47.229581 557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:47.229825 557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:47.231959 557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "cec60ec7446d4d3298c10678d614299f"
format_stamp: "Formatted at 2026-08-09 03:29:47 on dist-test-slave-65mx"
server_key: "14c528c9a87a5e8d6684908aae7f5393"
server_key_iv: "15e2c47fa2cf8ff4bc213ca71301d95f"
server_key_version: "encryptionkey@0"
I20260809 03:29:47.232636 557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "cec60ec7446d4d3298c10678d614299f"
format_stamp: "Formatted at 2026-08-09 03:29:47 on dist-test-slave-65mx"
server_key: "14c528c9a87a5e8d6684908aae7f5393"
server_key_iv: "15e2c47fa2cf8ff4bc213ca71301d95f"
server_key_version: "encryptionkey@0"
I20260809 03:29:47.236624 557 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:29:47.239148 573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:47.240422 557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:47.240624 557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "cec60ec7446d4d3298c10678d614299f"
format_stamp: "Formatted at 2026-08-09 03:29:47 on dist-test-slave-65mx"
server_key: "14c528c9a87a5e8d6684908aae7f5393"
server_key_iv: "15e2c47fa2cf8ff4bc213ca71301d95f"
server_key_version: "encryptionkey@0"
I20260809 03:29:47.240756 557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:47.262516 557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:47.265554 557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:47.265780 557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:47.273459 557 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:39849
I20260809 03:29:47.273491 628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:39849 every 8 connection(s)
I20260809 03:29:47.274492 557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:47.277608 629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:47.282853 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 557
I20260809 03:29:47.283023 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:47.283353 9403 external_mini_cluster.cc:1428] Setting key 3eef02e3825074a74caebaa0845579b9
I20260809 03:29:47.283798 629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f: Bootstrap starting.
I20260809 03:29:47.286444 629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:47.287236 629 log.cc:824] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:47.289330 629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f: No bootstrap required, opened a new log
Aug 09 03:29:47 dist-test-slave-65mx krb5kdc[543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:47.293784 629 raft_consensus.cc:348] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec60ec7446d4d3298c10678d614299f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39849 } }
I20260809 03:29:47.294104 629 raft_consensus.cc:374] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:47.294189 629 raft_consensus.cc:729] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec60ec7446d4d3298c10678d614299f, State: Initialized, Role: FOLLOWER
I20260809 03:29:47.294659 629 consensus_queue.cc:260] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec60ec7446d4d3298c10678d614299f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39849 } }
I20260809 03:29:47.294839 629 raft_consensus.cc:388] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:47.294932 629 raft_consensus.cc:482] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:47.295079 629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:47.296058 629 raft_consensus.cc:504] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec60ec7446d4d3298c10678d614299f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39849 } }
I20260809 03:29:47.296435 629 leader_election.cc:302] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cec60ec7446d4d3298c10678d614299f; no voters:
I20260809 03:29:47.296847 629 leader_election.cc:288] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:47.296939 635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:47.297228 635 raft_consensus.cc:686] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [term 1 LEADER]: Becoming Leader. State: Replica: cec60ec7446d4d3298c10678d614299f, State: Running, Role: LEADER
I20260809 03:29:47.297627 635 consensus_queue.cc:237] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec60ec7446d4d3298c10678d614299f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39849 } }
I20260809 03:29:47.298352 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:47.284881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45469 (local address 127.9.46.254:39849)
0809 03:29:47.285359 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:47.285374 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:47.285408 (+ 34us) server_negotiation.cc:407] Connection header received
0809 03:29:47.286024 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:47.286039 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:47.286372 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:47.286708 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:47.287565 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:47.288522 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:47.289436 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:47.289914 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:47.290106 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:47.293056 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:47.293061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:47.293074 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:47.293104 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:47.295143 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:47.295604 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:47.295611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:47.295617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:47.295750 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:47.296048 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:47.296052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:47.296053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:47.296538 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:47.296807 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:47.297130 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 03:29:47.297468 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":164,"threads_started":1}
I20260809 03:29:47.298403 629 sys_catalog.cc:564] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:47.298923 635 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cec60ec7446d4d3298c10678d614299f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec60ec7446d4d3298c10678d614299f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39849 } } }
I20260809 03:29:47.299117 635 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:47.300346 635 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec60ec7446d4d3298c10678d614299f. Latest consensus state: current_term: 1 leader_uuid: "cec60ec7446d4d3298c10678d614299f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec60ec7446d4d3298c10678d614299f" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39849 } } }
I20260809 03:29:47.300475 635 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:47.301712 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 03:29:47.303606 650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:29:47.303702 650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:29:47.303795 640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:47.304648 640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:47.311543 640 catalog_manager.cc:1347] Generated new cluster ID: 030be8ef1a454ff4b3e80da0dc9e08fe
I20260809 03:29:47.311617 640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:47.315883 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:47.305170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44911 (local address 127.9.46.254:39849)
0809 03:29:47.305299 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:47.305305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:47.305538 (+ 233us) server_negotiation.cc:407] Connection header received
0809 03:29:47.305680 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:47.305683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:47.305736 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:47.305832 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:47.306851 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:47.307703 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:47.308964 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:47.309372 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:47.309486 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:47.311074 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:47.311078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:47.311081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:47.311111 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:47.313966 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:47.314544 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:47.314547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:47.314549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:47.314638 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:47.314951 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:47.314953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:47.314955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:47.315174 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:47.315296 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:47.315566 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 03:29:47.315692 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 03:29:47.321192 640 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:47.322432 640 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:47.339725 640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cec60ec7446d4d3298c10678d614299f: Generated new TSK 0
I20260809 03:29:47.340636 640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:47.346745 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 557
2026-08-09T03:29:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:47 dist-test-slave-65mx krb5kdc[662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:47 dist-test-slave-65mx krb5kdc[662](info): set up 2 sockets
Aug 09 03:29:47 dist-test-slave-65mx krb5kdc[662](info): commencing operation
krb5kdc: starting...
W20260809 03:29:48.428777 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:48 dist-test-slave-65mx krb5kdc[662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:48.560009 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39575
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:38023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:39575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:48.671103 675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:48.671350 675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:48.671391 675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:48.675235 675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:48.675287 675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:48.675307 675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:48.675323 675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:48.675338 675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:48.680469 675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:38023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:39575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:48.681480 675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:48.682446 675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:48.691031 683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:48.691067 681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:48.691275 680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:48.691378 675 server_base.cc:1034] running on GCE node
I20260809 03:29:48.692265 675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:48.693670 675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:48.694876 675 hybrid_clock.cc:648] HybridClock initialized: now 1786246188694835 us; error 54 us; skew 500 ppm
Aug 09 03:29:48 dist-test-slave-65mx krb5kdc[662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246188, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:48.698369 675 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:48.699169 675 webserver.cc:460] Webserver started at http://127.9.46.254:40933/ using document root <none> and password file <none>
I20260809 03:29:48.699672 675 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:48.699731 675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:48.699892 675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:48.701923 675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "30551776fd054f0191fbfa02381d25ec"
format_stamp: "Formatted at 2026-08-09 03:29:48 on dist-test-slave-65mx"
server_key: "4e39692f09fc3757ea12e8b8a2e9f988"
server_key_iv: "72e1e3c6f7624ab18624fbbdc75bdb69"
server_key_version: "encryptionkey@0"
I20260809 03:29:48.702447 675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "30551776fd054f0191fbfa02381d25ec"
format_stamp: "Formatted at 2026-08-09 03:29:48 on dist-test-slave-65mx"
server_key: "4e39692f09fc3757ea12e8b8a2e9f988"
server_key_iv: "72e1e3c6f7624ab18624fbbdc75bdb69"
server_key_version: "encryptionkey@0"
I20260809 03:29:48.706065 675 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 03:29:48.708405 690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:48.709669 675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:48.709807 675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "30551776fd054f0191fbfa02381d25ec"
format_stamp: "Formatted at 2026-08-09 03:29:48 on dist-test-slave-65mx"
server_key: "4e39692f09fc3757ea12e8b8a2e9f988"
server_key_iv: "72e1e3c6f7624ab18624fbbdc75bdb69"
server_key_version: "encryptionkey@0"
I20260809 03:29:48.709913 675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:48.733381 675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:48.736769 675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:48.736999 675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:48.744665 675 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:39575
I20260809 03:29:48.744683 743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:39575 every 8 connection(s)
I20260809 03:29:48.745656 675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:48.747013 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 675
I20260809 03:29:48.747151 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:48.747429 9403 external_mini_cluster.cc:1428] Setting key 6413430523d61d7dc038c29288c3d3a2
I20260809 03:29:48.748674 744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:48.754359 744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec: Bootstrap starting.
I20260809 03:29:48.756728 748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:48.748845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55299 (local address 127.9.46.254:39575)
0809 03:29:48.749305 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:48.749321 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:48.749356 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:29:48.749990 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:48.750005 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:48.750335 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:48.750660 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:48.751107 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:48.752104 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:48.753124 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:48.753606 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:48.753784 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:48.754719 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:48.754726 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:48.754740 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:48.755215 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:48.755282 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:48.755314 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:48.755415 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:48.755569 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 03:29:48.755796 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":155,"threads_started":1}
I20260809 03:29:48.757160 744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:48.758004 744 log.cc:824] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:48.759717 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:48.760136 744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec: No bootstrap required, opened a new log
I20260809 03:29:48.763741 744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30551776fd054f0191fbfa02381d25ec" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39575 } }
I20260809 03:29:48.764019 744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:48.764081 744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30551776fd054f0191fbfa02381d25ec, State: Initialized, Role: FOLLOWER
I20260809 03:29:48.764520 744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30551776fd054f0191fbfa02381d25ec" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39575 } }
I20260809 03:29:48.764772 744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:48.764868 744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:48.764968 744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:48.766222 744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30551776fd054f0191fbfa02381d25ec" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39575 } }
I20260809 03:29:48.766466 748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:48.761855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57025 (local address 127.9.46.254:39575)
0809 03:29:48.761957 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:48.761960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:48.762155 (+ 195us) server_negotiation.cc:407] Connection header received
0809 03:29:48.762277 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:48.762279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:48.762315 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:48.762373 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:48.762664 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:48.763514 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:48.764637 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:48.765017 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:48.765137 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:48.765934 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:48.765937 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:48.765939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:48.765951 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:48.765975 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:48.766021 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:48.766087 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:48.766244 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 03:29:48.766290 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 03:29:48.766592 744 leader_election.cc:302] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30551776fd054f0191fbfa02381d25ec; no voters:
I20260809 03:29:48.766935 744 leader_election.cc:288] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:48.767078 756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:48.767314 756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [term 1 LEADER]: Becoming Leader. State: Replica: 30551776fd054f0191fbfa02381d25ec, State: Running, Role: LEADER
I20260809 03:29:48.767642 756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30551776fd054f0191fbfa02381d25ec" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39575 } }
I20260809 03:29:48.768190 744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:48.768813 757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30551776fd054f0191fbfa02381d25ec. Latest consensus state: current_term: 1 leader_uuid: "30551776fd054f0191fbfa02381d25ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30551776fd054f0191fbfa02381d25ec" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39575 } } }
I20260809 03:29:48.769078 757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:48.769302 758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30551776fd054f0191fbfa02381d25ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30551776fd054f0191fbfa02381d25ec" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39575 } } }
I20260809 03:29:48.769398 758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:48.769618 765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:48.772676 765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:48.778167 765 catalog_manager.cc:1347] Generated new cluster ID: 3aaa57728fe64e1885e7e7e07469463b
I20260809 03:29:48.778264 765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:48.788264 765 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:48.789218 765 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:48.796343 765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30551776fd054f0191fbfa02381d25ec: Generated new TSK 0
I20260809 03:29:48.797053 765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:48.805714 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 675
2026-08-09T03:29:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:48 dist-test-slave-65mx krb5kdc[779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:48 dist-test-slave-65mx krb5kdc[779](info): set up 2 sockets
Aug 09 03:29:48 dist-test-slave-65mx krb5kdc[779](info): commencing operation
krb5kdc: starting...
W20260809 03:29:49.885038 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:49 dist-test-slave-65mx krb5kdc[779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:50.013124 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39517
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:46503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:39517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:50.123921 793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:50.124293 793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:50.124390 793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:50.128402 793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:50.128492 793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:50.128532 793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:50.128587 793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:50.128620 793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:50.133867 793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:46503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:39517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:39517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:50.134892 793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:50.135902 793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:50.144371 799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:50.144418 793 server_base.cc:1034] running on GCE node
W20260809 03:29:50.144368 798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:50.144362 801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:50.145115 793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:50.146142 793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:50.147300 793 hybrid_clock.cc:648] HybridClock initialized: now 1786246190147275 us; error 37 us; skew 500 ppm
Aug 09 03:29:50 dist-test-slave-65mx krb5kdc[779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246190, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:50.150018 793 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:50.150897 793 webserver.cc:460] Webserver started at http://127.9.46.254:42547/ using document root <none> and password file <none>
I20260809 03:29:50.151392 793 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:50.151449 793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:50.151602 793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:50.153594 793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "8837e02c527c45baa68ad6343fe9cb41"
format_stamp: "Formatted at 2026-08-09 03:29:50 on dist-test-slave-65mx"
server_key: "ee5d75c4983c815774024c3ec17e5efe"
server_key_iv: "848d052303b7e5e766deaa4619f2ffbe"
server_key_version: "encryptionkey@0"
I20260809 03:29:50.154114 793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "8837e02c527c45baa68ad6343fe9cb41"
format_stamp: "Formatted at 2026-08-09 03:29:50 on dist-test-slave-65mx"
server_key: "ee5d75c4983c815774024c3ec17e5efe"
server_key_iv: "848d052303b7e5e766deaa4619f2ffbe"
server_key_version: "encryptionkey@0"
I20260809 03:29:50.157820 793 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 03:29:50.160172 809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:50.161516 793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 03:29:50.161655 793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "8837e02c527c45baa68ad6343fe9cb41"
format_stamp: "Formatted at 2026-08-09 03:29:50 on dist-test-slave-65mx"
server_key: "ee5d75c4983c815774024c3ec17e5efe"
server_key_iv: "848d052303b7e5e766deaa4619f2ffbe"
server_key_version: "encryptionkey@0"
I20260809 03:29:50.161765 793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:50.166335 793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:50.169227 793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:50.169411 793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:50.176164 793 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:39517
I20260809 03:29:50.176270 863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:39517 every 8 connection(s)
I20260809 03:29:50.177136 793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:50.179231 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 793
I20260809 03:29:50.179366 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:50.179672 9403 external_mini_cluster.cc:1428] Setting key c4775feeb216ab7d5e286614eb5474d4
I20260809 03:29:50.180023 866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:50.185088 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:50.181037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52575 (local address 127.9.46.254:39517)
0809 03:29:50.181489 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:50.181515 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:50.181560 (+ 45us) server_negotiation.cc:407] Connection header received
0809 03:29:50.182350 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:50.182370 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:50.182712 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:50.183061 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:50.183726 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:50.183731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:50.183745 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:50.183756 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:50.183814 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:50.183841 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:50.183980 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:50.184052 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 03:29:50.184204 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":153,"threads_started":1}
I20260809 03:29:50.187284 866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41: Bootstrap starting.
I20260809 03:29:50.187894 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:50.189674 866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:50.190618 866 log.cc:824] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:50.191764 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:50.190018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46105 (local address 127.9.46.254:39517)
0809 03:29:50.190144 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:50.190148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:50.190401 (+ 253us) server_negotiation.cc:407] Connection header received
0809 03:29:50.190527 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:50.190529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:50.190557 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:50.190621 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:50.191363 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:50.191366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:50.191367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 03:29:50.191377 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:50.191400 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 03:29:50.191433 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:50.191525 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:50.191620 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 03:29:50.191656 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 03:29:50.192929 866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41: No bootstrap required, opened a new log
I20260809 03:29:50.196380 866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8837e02c527c45baa68ad6343fe9cb41" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39517 } }
I20260809 03:29:50.196667 866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 03:29:50.196717 866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8837e02c527c45baa68ad6343fe9cb41, State: Initialized, Role: FOLLOWER
I20260809 03:29:50.197124 866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8837e02c527c45baa68ad6343fe9cb41" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39517 } }
I20260809 03:29:50.197239 866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 03:29:50.197285 866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 03:29:50.197352 866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:50.198365 866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8837e02c527c45baa68ad6343fe9cb41" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39517 } }
I20260809 03:29:50.198746 866 leader_election.cc:302] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8837e02c527c45baa68ad6343fe9cb41; no voters:
I20260809 03:29:50.199059 866 leader_election.cc:288] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 03:29:50.199180 877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:50.199409 877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [term 1 LEADER]: Becoming Leader. State: Replica: 8837e02c527c45baa68ad6343fe9cb41, State: Running, Role: LEADER
I20260809 03:29:50.199723 877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8837e02c527c45baa68ad6343fe9cb41" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39517 } }
I20260809 03:29:50.200217 866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:50.201339 879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8837e02c527c45baa68ad6343fe9cb41. Latest consensus state: current_term: 1 leader_uuid: "8837e02c527c45baa68ad6343fe9cb41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8837e02c527c45baa68ad6343fe9cb41" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39517 } } }
I20260809 03:29:50.201465 879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:50.201761 878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8837e02c527c45baa68ad6343fe9cb41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8837e02c527c45baa68ad6343fe9cb41" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 39517 } } }
I20260809 03:29:50.201843 878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:50.201851 886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:50.204735 886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:50.209622 886 catalog_manager.cc:1347] Generated new cluster ID: a3c98c4348ec4c63bf66517a4f15ac7f
I20260809 03:29:50.209695 886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:50.228217 886 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:50.228667 886 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:50.234268 886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8837e02c527c45baa68ad6343fe9cb41: Generated new TSK 0
I20260809 03:29:50.234978 886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 03:29:50.242264 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 793
2026-08-09T03:29:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1434 ms)
[----------] 18 tests from AuthTokenIssuingTest (26717 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:50 dist-test-slave-65mx krb5kdc[900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:50 dist-test-slave-65mx krb5kdc[900](info): set up 2 sockets
Aug 09 03:29:50 dist-test-slave-65mx krb5kdc[900](info): commencing operation
krb5kdc: starting...
W20260809 03:29:51.337138 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:51 dist-test-slave-65mx krb5kdc[900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:51.471580 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:44325
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:44325,127.9.46.253:36051,127.9.46.252:35393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:51.584363 914 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:29:51.584729 914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:51.584784 914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:51.584908 914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:51.588994 914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:51.589048 914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:51.589071 914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:51.589087 914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:51.589103 914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:51.594332 914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:44325,127.9.46.253:36051,127.9.46.252:35393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:44325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:51.595338 914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:51.596345 914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:51.604979 922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:51.604964 919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:51.605046 914 server_base.cc:1034] running on GCE node
W20260809 03:29:51.604981 920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:51.605784 914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:51.606974 914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:51.608165 914 hybrid_clock.cc:648] HybridClock initialized: now 1786246191608138 us; error 42 us; skew 500 ppm
Aug 09 03:29:51 dist-test-slave-65mx krb5kdc[900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:51.616215 914 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:51.617277 914 webserver.cc:460] Webserver started at http://127.9.46.254:34479/ using document root <none> and password file <none>
I20260809 03:29:51.617842 914 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:51.617908 914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:51.618083 914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:51.620141 914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "accf658a664e48d59b60032761ff7bfd"
format_stamp: "Formatted at 2026-08-09 03:29:51 on dist-test-slave-65mx"
server_key: "2b7eeacca9bd2b85e3e65ac5c2ea95aa"
server_key_iv: "83d6ae8f4c4ed0e8ba62a8fc4b6bc61c"
server_key_version: "encryptionkey@0"
I20260809 03:29:51.620759 914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "accf658a664e48d59b60032761ff7bfd"
format_stamp: "Formatted at 2026-08-09 03:29:51 on dist-test-slave-65mx"
server_key: "2b7eeacca9bd2b85e3e65ac5c2ea95aa"
server_key_iv: "83d6ae8f4c4ed0e8ba62a8fc4b6bc61c"
server_key_version: "encryptionkey@0"
I20260809 03:29:51.624604 914 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 03:29:51.627122 929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:51.628582 914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 03:29:51.628742 914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "accf658a664e48d59b60032761ff7bfd"
format_stamp: "Formatted at 2026-08-09 03:29:51 on dist-test-slave-65mx"
server_key: "2b7eeacca9bd2b85e3e65ac5c2ea95aa"
server_key_iv: "83d6ae8f4c4ed0e8ba62a8fc4b6bc61c"
server_key_version: "encryptionkey@0"
I20260809 03:29:51.628862 914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:51.645606 914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:51.649077 914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:51.649307 914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:51.656754 914 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:44325
I20260809 03:29:51.657058 986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:44325 every 8 connection(s)
I20260809 03:29:51.657804 914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:51.658380 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 914
I20260809 03:29:51.658540 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:51.658861 9403 external_mini_cluster.cc:1428] Setting key 0154c0e6839701afc9cc70efe8c0bf80
I20260809 03:29:51.660817 987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:51.664690 987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.46.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.253@KRBTEST.COM" created.
Aug 09 03:29:51 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:51.680008 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.666963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44325 (local address 127.0.0.1:43286)
0809 03:29:51.667533 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.667569 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:51.668414 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:51.669134 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:51.669139 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:51.669563 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:51.669849 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:51.669863 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.671134 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.671137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:51.671840 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:51.671849 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.672483 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.672487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:51.672631 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:51.673237 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:51.673254 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:51.675102 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:51.677155 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:51.677166 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:51.677176 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:51.677492 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:51.677832 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:51.677837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:51.677840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:51.677997 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:51.678609 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:51.678617 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:51.678790 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 03:29:51.678963 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 03:29:51.679306 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":99,"spinlock_wait_cycles":2816,"thread_start_us":140,"threads_started":1}
I20260809 03:29:51.680033 990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.666962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.9.46.254:44325)
0809 03:29:51.667392 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:51.667412 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:51.667675 (+ 263us) server_negotiation.cc:407] Connection header received
0809 03:29:51.668773 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:51.668778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:51.668944 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:51.669131 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:51.669997 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.670995 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.671958 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.672329 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.672520 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:51.675276 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:51.675281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:51.675289 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:51.675310 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:51.677034 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:51.677604 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:51.677608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:51.677624 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:51.677725 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:51.678118 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:51.678122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:51.678123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:51.678477 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:51.678581 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:51.679053 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 03:29:51.679223 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":280,"thread_start_us":143,"threads_started":1}
I20260809 03:29:51.682389 987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:51.683115 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.682872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:36051 (local address 127.0.0.1:53546)
0809 03:29:51.682990 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.683059 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:36051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 03:29:51.683320 933 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.46.253:36051: connect: Connection refused (error 111)
I20260809 03:29:51.684151 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.683954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:36051 (local address 127.0.0.1:53554)
0809 03:29:51.684058 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.684100 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:36051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 03:29:51.684310 987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.253:36051: Network error: Client connection negotiation failed: client connection to 127.9.46.253:36051: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab.
Entry for principal kudu/127.9.46.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab.
I20260809 03:29:51.686825 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.253:36051
--webserver_interface=127.9.46.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab
--principal=kudu/127.9.46.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:44325,127.9.46.253:36051,127.9.46.252:35393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:29:51.733507 987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } attempt: 1
I20260809 03:29:51.734398 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.734080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:36051 (local address 127.0.0.1:53564)
0809 03:29:51.734251 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.734320 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:36051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 03:29:51.735198 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.734923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:36051 (local address 127.0.0.1:53576)
0809 03:29:51.735091 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.735137 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:36051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260809 03:29:51.735342 987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.253:36051: Network error: Client connection negotiation failed: client connection to 127.9.46.253:36051: connect: Connection refused (error 111)
W20260809 03:29:51.801085 994 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:29:51.801457 994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:51.801535 994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:51.801622 994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 03:29:51.803508 987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } attempt: 2
I20260809 03:29:51.804325 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.804025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:36051 (local address 127.0.0.1:53580)
0809 03:29:51.804185 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.804253 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:36051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 03:29:51.805017 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.804808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:36051 (local address 127.0.0.1:53588)
0809 03:29:51.804915 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.804967 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:36051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 03:29:51.805234 987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.253:36051: Network error: Client connection negotiation failed: client connection to 127.9.46.253:36051: connect: Connection refused (error 111)
W20260809 03:29:51.806033 994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:51.806113 994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:51.806160 994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:51.806196 994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:51.806247 994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:51.811684 994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:44325,127.9.46.253:36051,127.9.46.252:35393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab
--principal=kudu/127.9.46.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.253:36051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.46.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:51.812858 994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:51.813872 994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:51.822211 999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:51.822204 1000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:51.822204 1002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:51.822618 994 server_base.cc:1034] running on GCE node
I20260809 03:29:51.823156 994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:51.824190 994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:51.825407 994 hybrid_clock.cc:648] HybridClock initialized: now 1786246191825383 us; error 50 us; skew 500 ppm
Aug 09 03:29:51 dist-test-slave-65mx krb5kdc[900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:51.828246 994 init.cc:377] Logged in from keytab as kudu/127.9.46.253@KRBTEST.COM (short username kudu)
I20260809 03:29:51.829198 994 webserver.cc:460] Webserver started at http://127.9.46.253:41715/ using document root <none> and password file <none>
I20260809 03:29:51.829764 994 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:51.829864 994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:51.830111 994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:51.832304 994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/data/instance:
uuid: "b17d4e94c9a1417489b3356be3fc8cf1"
format_stamp: "Formatted at 2026-08-09 03:29:51 on dist-test-slave-65mx"
server_key: "0ec889060eed6016026cfac066c92c5d"
server_key_iv: "ce0cb5861f19066a817fcd0ac3c46741"
server_key_version: "encryptionkey@0"
I20260809 03:29:51.832919 994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/wal/instance:
uuid: "b17d4e94c9a1417489b3356be3fc8cf1"
format_stamp: "Formatted at 2026-08-09 03:29:51 on dist-test-slave-65mx"
server_key: "0ec889060eed6016026cfac066c92c5d"
server_key_iv: "ce0cb5861f19066a817fcd0ac3c46741"
server_key_version: "encryptionkey@0"
I20260809 03:29:51.836745 994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 03:29:51.839164 1010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:51.840454 994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 03:29:51.840641 994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/wal
uuid: "b17d4e94c9a1417489b3356be3fc8cf1"
format_stamp: "Formatted at 2026-08-09 03:29:51 on dist-test-slave-65mx"
server_key: "0ec889060eed6016026cfac066c92c5d"
server_key_iv: "ce0cb5861f19066a817fcd0ac3c46741"
server_key_version: "encryptionkey@0"
I20260809 03:29:51.840767 994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:51.860350 994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:51.863341 994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:51.863561 994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:51.870942 994 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.253:36051
I20260809 03:29:51.871237 1065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.253:36051 every 8 connection(s)
I20260809 03:29:51.871906 994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
I20260809 03:29:51.873900 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 994
I20260809 03:29:51.874035 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-1/wal/instance
I20260809 03:29:51.874348 9403 external_mini_cluster.cc:1428] Setting key 24e2a32c24c74a3c2846d0ea4ce30677
I20260809 03:29:51.875070 1066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:51.879196 1066 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.46.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.252@KRBTEST.COM" created.
Aug 09 03:29:51 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:51.894546 990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.881145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.9.46.254:44325)
0809 03:29:51.881335 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:51.881341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:51.882213 (+ 872us) server_negotiation.cc:407] Connection header received
0809 03:29:51.883545 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:51.883548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:51.883598 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:51.883675 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:51.884940 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.885715 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.887075 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.887526 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.887653 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:51.890446 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:51.890453 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:51.890459 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:51.890498 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:51.892126 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:51.892841 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:51.892844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:51.892845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:51.892922 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:51.893356 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:51.893359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:51.893360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:51.893533 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:51.893607 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:51.894311 (+ 704us) server_negotiation.cc:299] Negotiation successful
0809 03:29:51.894418 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 03:29:51.895830 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.881465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44325 (local address 127.0.0.1:43290)
0809 03:29:51.882005 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.882072 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:51.883299 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:51.883864 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:51.883871 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:51.884196 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:51.884808 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:51.884821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.885841 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.885844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:51.886942 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:51.886953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.887681 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.887684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:51.887818 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:51.888487 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:51.888517 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:51.890265 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:51.892255 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:51.892265 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:51.892285 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:51.892739 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:51.893022 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:51.893025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:51.893027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:51.893242 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:51.893644 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:51.893650 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:51.894012 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 03:29:51.894251 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 03:29:51.894626 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":119,"threads_started":1}
I20260809 03:29:51.896400 987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } attempt: 3
I20260809 03:29:51.897714 1066 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:29:51 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab.
Entry for principal kudu/127.9.46.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab.
Aug 09 03:29:51 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
I20260809 03:29:51.904448 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.252:35393
--webserver_interface=127.9.46.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab
--principal=kudu/127.9.46.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:44325,127.9.46.253:36051,127.9.46.252:35393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:29:51.907732 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.898188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:36051 (local address 127.0.0.1:53610)
0809 03:29:51.898304 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.898318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:51.898392 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:51.898823 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:51.898825 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:51.898998 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:51.899081 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:51.899087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.899866 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.899872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:51.900391 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:51.900399 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.901116 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.901120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:51.901331 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:51.902010 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:51.902030 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:51.903818 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:51.906011 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:51.906014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:51.906016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:51.906265 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:51.906602 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:51.906605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:51.906607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:51.906709 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:51.907286 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:51.907289 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:51.907357 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 03:29:51.907457 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 03:29:51.907584 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":109}
I20260809 03:29:51.908088 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.896878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:36051 (local address 127.0.0.1:53594)
0809 03:29:51.897027 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.897042 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:51.897132 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:51.897849 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:51.897854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:51.898034 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:51.898165 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:51.898173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.899278 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.899283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:51.900429 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:51.900439 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.901122 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.901126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:51.901243 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:51.901843 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:51.901858 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:51.903243 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:51.906310 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:51.906316 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:51.906320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:51.906615 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:51.906918 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:51.906921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:51.906923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:51.907009 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:51.907461 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:51.907466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:51.907582 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 03:29:51.907698 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 03:29:51.907964 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 03:29:51.908376 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.898335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53610 (local address 127.9.46.253:36051)
0809 03:29:51.898642 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:51.898648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:51.898658 (+ 10us) server_negotiation.cc:407] Connection header received
0809 03:29:51.898684 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:51.898685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:51.898715 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:51.898815 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:51.899191 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.899766 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.900508 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.900999 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.901119 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:51.903958 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:51.903964 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:51.903969 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:51.903999 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:51.905885 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:51.906396 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:51.906403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:51.906406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:51.906496 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:51.906809 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:51.906813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:51.906816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:51.907170 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:51.907311 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:51.907932 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 03:29:51.908110 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":243,"spinlock_wait_cycles":7936,"thread_start_us":93,"threads_started":1}
I20260809 03:29:51.908841 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.897019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53594 (local address 127.9.46.253:36051)
0809 03:29:51.897399 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:51.897410 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:51.897431 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:29:51.897470 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:51.897476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:51.897709 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:51.897921 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:51.898261 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.899115 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.900571 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:51.901001 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:51.901178 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:51.903453 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:51.903466 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:51.903483 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:51.903515 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:51.906169 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:51.906735 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:51.906741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:51.906746 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:51.906824 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:51.907114 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:51.907119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:51.907120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:51.907347 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:51.907451 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:51.908622 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0809 03:29:51.908730 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":285,"thread_start_us":184,"threads_started":1}
I20260809 03:29:51.910238 1066 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:51.910259 987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:51.910969 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.910704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36048)
0809 03:29:51.910852 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.910907 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 03:29:51.910969 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.910704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36036)
0809 03:29:51.910852 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.910919 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 03:29:51.911155 1013 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
I20260809 03:29:51.911880 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.911439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36056)
0809 03:29:51.911758 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.911809 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":90,"threads_started":1}
W20260809 03:29:51.912068 987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:35393: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
I20260809 03:29:51.912220 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.911828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36068)
0809 03:29:51.912023 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.912100 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 03:29:51.912379 1066 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:35393: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
I20260809 03:29:51.943238 987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } attempt: 1
I20260809 03:29:51.944221 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.943803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36074)
0809 03:29:51.943990 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.944089 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 03:29:51.945134 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.944853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36082)
0809 03:29:51.944992 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.945051 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 03:29:51.945294 987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:35393: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
I20260809 03:29:51.961676 1066 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } attempt: 1
I20260809 03:29:51.962642 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.962300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36098)
0809 03:29:51.962466 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.962541 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 03:29:51.963443 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:51.963220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36112)
0809 03:29:51.963336 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:51.963389 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 03:29:51.963588 1066 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:35393: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
I20260809 03:29:52.020478 987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } attempt: 2
I20260809 03:29:52.021361 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.021017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36120)
0809 03:29:52.021201 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:52.021270 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 03:29:52.022114 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.021882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36124)
0809 03:29:52.021993 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:52.022045 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 03:29:52.022271 987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:35393: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
W20260809 03:29:52.023013 1073 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:29:52.023307 1073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:52.023356 1073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:52.023442 1073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:52.027535 1073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:52.027598 1073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:52.027618 1073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:52.027671 1073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:52.027702 1073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:52.031800 1066 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } attempt: 2
I20260809 03:29:52.032696 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.032370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36126)
0809 03:29:52.032527 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:52.032612 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 03:29:52.033511 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.033263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36134)
0809 03:29:52.033377 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:52.033448 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 03:29:52.033694 1066 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:35393: Network error: Client connection negotiation failed: client connection to 127.9.46.252:35393: connect: Connection refused (error 111)
I20260809 03:29:52.033804 1073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:44325,127.9.46.253:36051,127.9.46.252:35393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab
--principal=kudu/127.9.46.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.252:35393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.46.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:52.034921 1073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:52.035943 1073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:52.044648 1083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:52.044759 1080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:52.044832 1081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:52.044862 1073 server_base.cc:1034] running on GCE node
I20260809 03:29:52.045862 1073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:52.046980 1073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:52.048167 1073 hybrid_clock.cc:648] HybridClock initialized: now 1786246192048155 us; error 28 us; skew 500 ppm
Aug 09 03:29:52 dist-test-slave-65mx krb5kdc[900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:52.050984 1073 init.cc:377] Logged in from keytab as kudu/127.9.46.252@KRBTEST.COM (short username kudu)
I20260809 03:29:52.051865 1073 webserver.cc:460] Webserver started at http://127.9.46.252:34385/ using document root <none> and password file <none>
I20260809 03:29:52.052392 1073 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:52.052450 1073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:52.052695 1073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:52.054718 1073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/data/instance:
uuid: "703a4e08902545789dacd004d4738b72"
format_stamp: "Formatted at 2026-08-09 03:29:52 on dist-test-slave-65mx"
server_key: "4b0246e2e19e76060f894b08a2c9fb8c"
server_key_iv: "180b32a88343837b201d44a656a17cff"
server_key_version: "encryptionkey@0"
I20260809 03:29:52.055253 1073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/wal/instance:
uuid: "703a4e08902545789dacd004d4738b72"
format_stamp: "Formatted at 2026-08-09 03:29:52 on dist-test-slave-65mx"
server_key: "4b0246e2e19e76060f894b08a2c9fb8c"
server_key_iv: "180b32a88343837b201d44a656a17cff"
server_key_version: "encryptionkey@0"
I20260809 03:29:52.059118 1073 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 03:29:52.061506 1090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:52.062779 1073 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:52.062914 1073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/wal
uuid: "703a4e08902545789dacd004d4738b72"
format_stamp: "Formatted at 2026-08-09 03:29:52 on dist-test-slave-65mx"
server_key: "4b0246e2e19e76060f894b08a2c9fb8c"
server_key_iv: "180b32a88343837b201d44a656a17cff"
server_key_version: "encryptionkey@0"
I20260809 03:29:52.063026 1073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:52.090174 1073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:52.093320 1073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:52.093502 1073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:52.101095 1073 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.252:35393
I20260809 03:29:52.101421 1148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.252:35393 every 8 connection(s)
I20260809 03:29:52.102270 1073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
I20260809 03:29:52.103956 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 1073
I20260809 03:29:52.104110 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786245904130257-9403-0/minicluster-data/master-2/wal/instance
I20260809 03:29:52.104415 9403 external_mini_cluster.cc:1428] Setting key 61286cc8cbb45c2c25a3612288e3d1a6
I20260809 03:29:52.105355 1149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:52.110140 1149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:29:52 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
Aug 09 03:29:52 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:52.120945 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.105847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36144 (local address 127.9.46.252:35393)
0809 03:29:52.106308 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.106327 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.106369 (+ 42us) server_negotiation.cc:407] Connection header received
0809 03:29:52.107263 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.107287 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.107697 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.108062 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.108970 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.110457 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.111359 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.111925 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.112108 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.115341 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.115346 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.115355 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.115383 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.117449 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.117917 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.117923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.117929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.118046 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.118458 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.118460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.118461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.118986 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.119146 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.119462 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.119823 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":134,"threads_started":1}
I20260809 03:29:52.121405 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.111938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:44325 (local address 127.0.0.1:43296)
0809 03:29:52.112382 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:52.112459 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:52.112619 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:52.112986 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:52.112990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:52.113336 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:52.113459 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:52.113465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.114424 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.114428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:52.115158 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:52.115168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.115637 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.115640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:52.115764 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.116204 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:52.116216 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:52.117672 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:52.119631 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:52.119638 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:52.119644 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:52.119960 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:52.120243 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:52.120245 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:52.120247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:52.120337 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:52.120755 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:52.120761 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:52.120877 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 03:29:52.121032 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 03:29:52.121225 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":188,"threads_started":1}
I20260809 03:29:52.121428 987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } attempt: 3
I20260809 03:29:52.121423 990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.112008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.9.46.254:44325)
0809 03:29:52.112142 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.112146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.112579 (+ 433us) server_negotiation.cc:407] Connection header received
0809 03:29:52.112782 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.112788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.112846 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.112919 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.113603 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.114287 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.115293 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.115514 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.115640 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.117825 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.117828 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.117831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.117855 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.119520 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.120060 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.120063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.120064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.120147 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.120426 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.120428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.120429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.120654 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.120732 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.121158 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.121275 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 03:29:52.122850 1149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:52.124871 1066 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } attempt: 3
Aug 09 03:29:52 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
Aug 09 03:29:52 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
Aug 09 03:29:52 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
I20260809 03:29:52.132217 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.121836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36146)
0809 03:29:52.121985 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:52.122003 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:52.122101 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:52.122585 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:52.122588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:52.122776 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:52.122924 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:52.122933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.124269 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.124271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:52.124960 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:52.124991 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.125652 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.125656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:52.125751 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.126319 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:52.126338 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:52.128144 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:52.130352 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:52.130356 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:52.130358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:52.130620 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:52.130992 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:52.130997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:52.131001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:52.131109 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:52.131622 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:52.131625 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:52.131730 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 03:29:52.131825 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 03:29:52.131989 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 03:29:52.133061 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.122074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36146 (local address 127.9.46.252:35393)
0809 03:29:52.122371 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.122380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.122392 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:29:52.122426 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.122429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.122467 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.122553 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.123051 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.124141 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.125126 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.125500 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.125990 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.128277 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.128283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.128285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.128337 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.130221 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.130762 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.130765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.130767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.130859 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.131249 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.131254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.131256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.131497 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.132536 (+ 1039us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.132637 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.132829 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":221,"spinlock_wait_cycles":47104,"thread_start_us":106,"threads_started":1}
Aug 09 03:29:52 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:52.134087 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.123456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53616 (local address 127.9.46.253:36051)
0809 03:29:52.123762 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.123767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.123782 (+ 15us) server_negotiation.cc:407] Connection header received
0809 03:29:52.123823 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.123825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.123867 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.123951 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.124364 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.124989 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.126748 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.127137 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.127216 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.130097 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.130103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.130109 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.130142 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.131995 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.132534 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.132537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.132572 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.132666 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.132983 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.132986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.132988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.133354 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.133429 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.133797 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.133931 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208}
I20260809 03:29:52.134613 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.123332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:36051 (local address 127.0.0.1:53616)
0809 03:29:52.123612 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:52.123626 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:52.123701 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:52.123972 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:52.123974 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:52.124112 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:52.124248 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:52.124257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.125126 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.125130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:52.126607 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:52.126618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.127374 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.127377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:52.127478 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.128308 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:52.128330 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:52.129900 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:52.132144 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:52.132147 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:52.132148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:52.132421 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:52.132790 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:52.132795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:52.132796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:52.132878 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:52.133502 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:52.133505 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:52.133610 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 03:29:52.134276 (+ 666us) client_negotiation.cc:240] Negotiation successful
0809 03:29:52.134415 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":4,"spinlock_wait_cycles":896}
I20260809 03:29:52.135277 1149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:52.136233 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.125279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36160)
0809 03:29:52.125518 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:52.125533 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:52.125624 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:52.126150 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:52.126154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:52.126348 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:52.126474 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:52.126482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.127507 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.127510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:52.128336 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:52.128347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.129114 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.129118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:52.129201 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.129642 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:52.129657 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:52.131259 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:52.134053 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:52.134056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:52.134058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:52.134334 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:52.135130 (+ 796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:52.135134 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:52.135137 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:52.135242 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:52.135710 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:52.135715 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:52.135821 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 03:29:52.135906 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 03:29:52.136048 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260809 03:29:52.136857 987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd: Bootstrap starting.
I20260809 03:29:52.136902 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.125788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36160 (local address 127.9.46.252:35393)
0809 03:29:52.125913 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.125917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.125930 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:29:52.125968 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.125973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.126014 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.126592 (+ 578us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.126672 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.127389 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.128583 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.128980 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.129425 (+ 445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.131427 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.131431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.131433 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.131476 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.133932 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.134904 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.134910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.134913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.135000 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.135366 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.135372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.135373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.135578 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.136513 (+ 935us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.136624 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.136746 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55,"spinlock_wait_cycles":6912}
I20260809 03:29:52.138638 990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.123624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.9.46.254:44325)
0809 03:29:52.123720 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.123727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.124184 (+ 457us) server_negotiation.cc:407] Connection header received
0809 03:29:52.125163 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.125169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.125215 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.125284 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.126504 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.127260 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.128980 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.129740 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.129847 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.134087 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.134093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.134099 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.134126 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.136246 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.136820 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.136824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.136826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.136907 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.137287 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.137290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.137291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.137520 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.137700 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.138382 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.138492 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 03:29:52.139834 1066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1: Bootstrap starting.
I20260809 03:29:52.139833 987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:52.140790 987 log.cc:824] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd: Log is configured to *not* fsync() on all Append() calls
Aug 09 03:29:52 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246192, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
I20260809 03:29:52.142902 987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd: No bootstrap required, opened a new log
I20260809 03:29:52.142982 1066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:52.144084 1066 log.cc:824] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:52.146497 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.135792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:35393 (local address 127.0.0.1:36176)
0809 03:29:52.136178 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:52.136192 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:52.136265 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:52.136937 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:52.136940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:52.137111 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:52.137201 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:52.137207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.138488 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.138491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:52.139200 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:52.139210 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.139974 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.139977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:52.140062 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.140591 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:52.140606 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:52.142340 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:52.144792 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:52.144795 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:52.144797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:52.145058 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:52.145407 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:52.145410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:52.145412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:52.145498 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:52.145964 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:52.145966 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:52.146058 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 03:29:52.146172 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 03:29:52.146320 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326}
I20260809 03:29:52.146719 1066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1: No bootstrap required, opened a new log
I20260809 03:29:52.147022 1155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.135974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36176 (local address 127.9.46.252:35393)
0809 03:29:52.136725 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.136733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.136745 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:29:52.136775 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.136777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.136816 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.137507 (+ 691us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.137632 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.138374 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.139444 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.139833 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.140018 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.142502 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.142505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.142508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.142534 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.144677 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.145179 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.145182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.145184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.145286 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.145601 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.145604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.145606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.145829 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.145924 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.146719 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.146850 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":101,"threads_started":1}
I20260809 03:29:52.147428 987 raft_consensus.cc:348] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } }
I20260809 03:29:52.147696 987 raft_consensus.cc:729] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: accf658a664e48d59b60032761ff7bfd, State: Initialized, Role: FOLLOWER
I20260809 03:29:52.148061 987 consensus_queue.cc:260] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } }
I20260809 03:29:52.149592 987 sys_catalog.cc:564] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:52.150313 1149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72: Bootstrap starting.
W20260809 03:29:52.151189 935 tablet.cc:2369] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 03:29:52.151196 1066 raft_consensus.cc:348] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } }
I20260809 03:29:52.151443 1066 raft_consensus.cc:729] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17d4e94c9a1417489b3356be3fc8cf1, State: Initialized, Role: FOLLOWER
I20260809 03:29:52.151805 1066 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } }
I20260809 03:29:52.153239 1149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:52.153342 1066 sys_catalog.cc:564] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:52.154261 1149 log.cc:824] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:52.156358 1166 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } } }
I20260809 03:29:52.156585 1166 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 03:29:52.156586 1172 catalog_manager.cc:1558] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:29:52.156653 1172 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:29:52.156824 1158 sys_catalog.cc:455] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } } }
I20260809 03:29:52.156973 1158 sys_catalog.cc:458] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:52.155843 914 master_runner.cc:186] Error getting master registration for 127.9.46.253:36051: OK, instance_id { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" instance_seqno: 1786246191868947 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:29:52.159087 914 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 03:29:52 dist-test-slave-65mx krb5kdc[900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
W20260809 03:29:52.163829 1185 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:29:52.163990 1185 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:29:52.165043 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.146026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53620 (local address 127.9.46.253:36051)
0809 03:29:52.146153 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.146158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.146171 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:29:52.146295 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.146298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.146342 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.146393 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.148608 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.149406 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.150711 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.151056 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.151152 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.161068 (+ 9916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.161073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.161075 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.161116 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.162907 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.163470 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.163474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.163476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.163564 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.163883 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.163886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.163888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.164155 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.164333 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.164777 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.164903 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 03:29:52.162096 994 master_runner.cc:186] Error getting master registration for 127.9.46.252:35393: OK, instance_id { permanent_uuid: "703a4e08902545789dacd004d4738b72" instance_seqno: 1786246192098994 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:29:52.166121 1149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72: No bootstrap required, opened a new log
I20260809 03:29:52.166132 994 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:29:52.172245 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:52.173584 957 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "accf658a664e48d59b60032761ff7bfd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43298
I20260809 03:29:52.173938 957 raft_consensus.cc:482] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 03:29:52.174057 957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:52.173904 1149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } }
I20260809 03:29:52.174175 1149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 703a4e08902545789dacd004d4738b72, State: Initialized, Role: FOLLOWER
I20260809 03:29:52.174513 1149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } }
I20260809 03:29:52.175499 957 raft_consensus.cc:504] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } }
I20260809 03:29:52.175979 1149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:52.177014 957 leader_election.cc:288] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [CANDIDATE]: Term 1 election: Requested vote from peers b17d4e94c9a1417489b3356be3fc8cf1 (127.9.46.253:36051), 703a4e08902545789dacd004d4738b72 (127.9.46.252:35393)
I20260809 03:29:52.177102 1038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "accf658a664e48d59b60032761ff7bfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b17d4e94c9a1417489b3356be3fc8cf1"
I20260809 03:29:52.177345 1038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:52.178473 1038 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate accf658a664e48d59b60032761ff7bfd in term 1.
I20260809 03:29:52.178879 933 leader_election.cc:302] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: accf658a664e48d59b60032761ff7bfd, b17d4e94c9a1417489b3356be3fc8cf1; no voters:
I20260809 03:29:52.179273 1158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:52.179235 1120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "accf658a664e48d59b60032761ff7bfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "703a4e08902545789dacd004d4738b72"
I20260809 03:29:52.179450 1120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:52.179644 1158 raft_consensus.cc:686] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [term 1 LEADER]: Becoming Leader. State: Replica: accf658a664e48d59b60032761ff7bfd, State: Running, Role: LEADER
I20260809 03:29:52.180028 1158 consensus_queue.cc:237] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } }
I20260809 03:29:52.180840 1120 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate accf658a664e48d59b60032761ff7bfd in term 1.
I20260809 03:29:52.182159 1158 sys_catalog.cc:455] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader accf658a664e48d59b60032761ff7bfd. Latest consensus state: current_term: 1 leader_uuid: "accf658a664e48d59b60032761ff7bfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } } }
I20260809 03:29:52.182361 1158 sys_catalog.cc:458] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:52.184731 1187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } } }
I20260809 03:29:52.184875 1187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 03:29:52.187597 1206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:29:52.187722 1206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:29:52.192013 1195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:52.193713 1195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:52.198169 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.186443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.9.46.252:35393)
0809 03:29:52.186565 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.186570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.188853 (+ 2283us) server_negotiation.cc:407] Connection header received
0809 03:29:52.189055 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.189058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.189120 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.189173 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.190345 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.190899 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.191781 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.192097 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.192203 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.194219 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.194223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.194226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.194255 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.196041 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.196536 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.196583 (+ 47us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.196585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.196688 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.197130 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.197135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.197137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.197406 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.197516 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.197907 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.198036 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 03:29:52.205765 990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.187960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.9.46.254:44325)
0809 03:29:52.188103 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.188107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.194240 (+ 6133us) server_negotiation.cc:407] Connection header received
0809 03:29:52.194417 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.194420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.194484 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.194552 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.197766 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.198610 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.199871 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.200083 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.200233 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.202108 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.202112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.202115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.202144 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.203883 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.204381 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.204385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.204386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.204470 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.204794 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.204797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.204799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.205010 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.205144 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.205483 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.205601 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 03:29:52.206702 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:52.187285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.9.46.253:36051)
0809 03:29:52.187402 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.187406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.192818 (+ 5412us) server_negotiation.cc:407] Connection header received
0809 03:29:52.193003 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.193006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.193058 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.193112 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.195438 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.196305 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.199908 (+ 3603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:52.200298 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:52.200378 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:52.202308 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:52.202312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:52.202315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:52.202347 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:52.204745 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.205289 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.205293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.205295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.205384 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:52.205714 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:52.205721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:52.205725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:52.205944 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:52.206053 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:52.206403 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 03:29:52.206526 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 03:29:52.212659 1120 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [term 1 FOLLOWER]: Refusing update from remote peer accf658a664e48d59b60032761ff7bfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:52.212839 1038 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [term 1 FOLLOWER]: Refusing update from remote peer accf658a664e48d59b60032761ff7bfd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:52.213514 1158 consensus_queue.cc:1035] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:52.213960 1158 consensus_queue.cc:1035] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:52.218679 1166 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader accf658a664e48d59b60032761ff7bfd. Latest consensus state: current_term: 1 leader_uuid: "accf658a664e48d59b60032761ff7bfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } } }
I20260809 03:29:52.218801 1166 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:52.220063 1187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader accf658a664e48d59b60032761ff7bfd. Latest consensus state: current_term: 1 leader_uuid: "accf658a664e48d59b60032761ff7bfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } } }
I20260809 03:29:52.220198 1187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:52.223453 1189 sys_catalog.cc:455] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "accf658a664e48d59b60032761ff7bfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } } }
I20260809 03:29:52.223593 1189 sys_catalog.cc:458] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:52.224444 1211 mvcc.cc:205] Tried to move back new op lower bound from 7316464403293835264 to 7316464403174047744. Current Snapshot: MvccSnapshot[applied={T|T < 7316464403293835264}]
I20260809 03:29:52.224947 1166 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "accf658a664e48d59b60032761ff7bfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } } }
I20260809 03:29:52.225060 1166 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:52.225718 1212 mvcc.cc:205] Tried to move back new op lower bound from 7316464403293835264 to 7316464403174047744. Current Snapshot: MvccSnapshot[applied={T|T < 7316464403293835264}]
I20260809 03:29:52.230202 1195 catalog_manager.cc:1347] Generated new cluster ID: 9c9b618a9c754af99cf006cea78a6639
I20260809 03:29:52.230278 1195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:52.234624 1189 sys_catalog.cc:455] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "accf658a664e48d59b60032761ff7bfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } } }
I20260809 03:29:52.234747 1189 sys_catalog.cc:458] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:52.235693 1187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "accf658a664e48d59b60032761ff7bfd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "accf658a664e48d59b60032761ff7bfd" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 44325 } } peers { permanent_uuid: "b17d4e94c9a1417489b3356be3fc8cf1" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 36051 } } peers { permanent_uuid: "703a4e08902545789dacd004d4738b72" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 35393 } } }
I20260809 03:29:52.235812 1187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:52.236224 1214 mvcc.cc:205] Tried to move back new op lower bound from 7316464403293835264 to 7316464403174047744. Current Snapshot: MvccSnapshot[applied={T|T < 7316464403293835264}]
I20260809 03:29:52.254926 1195 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:52.256209 1195 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:52.269615 1195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P accf658a664e48d59b60032761ff7bfd: Generated new TSK 0
I20260809 03:29:52.270535 1195 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:29:52.308279 990 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.304949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.9.46.254:44325)
0809 03:29:52.305148 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.305153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.307128 (+ 1975us) server_negotiation.cc:407] Connection header received
0809 03:29:52.307349 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.307353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.307404 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.307458 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.308135 (+ 677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43312: BlockingRecv error: recv got EOF from 127.0.0.1:43312 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 03:29:52.309765 1152 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.305505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.9.46.252:35393)
0809 03:29:52.305659 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.305664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.308820 (+ 3156us) server_negotiation.cc:407] Connection header received
0809 03:29:52.308996 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.309002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.309066 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.309119 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.309641 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36206: BlockingRecv error: recv got EOF from 127.0.0.1:36206 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 03:29:52.311136 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.305903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.9.46.253:36051)
0809 03:29:52.306066 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.306071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.310245 (+ 4174us) server_negotiation.cc:407] Connection header received
0809 03:29:52.310414 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.310417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.310472 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.310526 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.311009 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53630: BlockingRecv error: recv got EOF from 127.0.0.1:53630 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 03:29:52.315837 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.314311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.9.46.253:36051)
0809 03:29:52.314417 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.314421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.314867 (+ 446us) server_negotiation.cc:407] Connection header received
0809 03:29:52.315102 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.315105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.315180 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.315233 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.315736 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53644: BlockingRecv error: recv got EOF from 127.0.0.1:53644 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 03:29:52.320847 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.319402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53656 (local address 127.9.46.253:36051)
0809 03:29:52.319495 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.319499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.319972 (+ 473us) server_negotiation.cc:407] Connection header received
0809 03:29:52.320158 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.320162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.320208 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.320256 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.320752 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53656: BlockingRecv error: recv got EOF from 127.0.0.1:53656 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 03:29:52.326612 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.325083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.9.46.253:36051)
0809 03:29:52.325178 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.325182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.325716 (+ 534us) server_negotiation.cc:407] Connection header received
0809 03:29:52.325890 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.325893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.325941 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.325991 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.326511 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53658: BlockingRecv error: recv got EOF from 127.0.0.1:53658 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 03:29:52.334674 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.332802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53674 (local address 127.9.46.253:36051)
0809 03:29:52.332944 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.332948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.333557 (+ 609us) server_negotiation.cc:407] Connection header received
0809 03:29:52.333877 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.333881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.333938 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.333993 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.334570 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53674: BlockingRecv error: recv got EOF from 127.0.0.1:53674 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 03:29:52.347384 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.345376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53678 (local address 127.9.46.253:36051)
0809 03:29:52.345512 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.345517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.346294 (+ 777us) server_negotiation.cc:407] Connection header received
0809 03:29:52.346516 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.346520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.346579 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.346639 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.347263 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53678: BlockingRecv error: recv got EOF from 127.0.0.1:53678 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 03:29:52.368286 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.366340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.9.46.253:36051)
0809 03:29:52.366472 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.366477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.367136 (+ 659us) server_negotiation.cc:407] Connection header received
0809 03:29:52.367379 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.367383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.367448 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.367509 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.368162 (+ 653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53684: BlockingRecv error: recv got EOF from 127.0.0.1:53684 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 03:29:52.404717 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.403278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53688 (local address 127.9.46.253:36051)
0809 03:29:52.403412 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.403416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.403739 (+ 323us) server_negotiation.cc:407] Connection header received
0809 03:29:52.403913 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.403916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.403973 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.404063 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.404592 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53688: BlockingRecv error: recv got EOF from 127.0.0.1:53688 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 03:29:52.472311 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.471023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53694 (local address 127.9.46.253:36051)
0809 03:29:52.471188 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.471191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.471297 (+ 106us) server_negotiation.cc:407] Connection header received
0809 03:29:52.471419 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.471421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.471479 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.471564 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.472145 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53694: BlockingRecv error: recv got EOF from 127.0.0.1:53694 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 03:29:52.603610 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.602458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53696 (local address 127.9.46.253:36051)
0809 03:29:52.602636 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.602643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.602705 (+ 62us) server_negotiation.cc:407] Connection header received
0809 03:29:52.602857 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.602860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.602925 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.603015 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.603482 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53696: BlockingRecv error: recv got EOF from 127.0.0.1:53696 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 03:29:52.862917 1071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:52.861919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53708 (local address 127.9.46.253:36051)
0809 03:29:52.862067 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:52.862070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:52.862155 (+ 85us) server_negotiation.cc:407] Connection header received
0809 03:29:52.862277 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:52.862279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:52.862328 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:52.862409 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:52.862821 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53708: BlockingRecv error: recv got EOF from 127.0.0.1:53708 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:29:53.165668 1185 catalog_manager.cc:1259] Loaded cluster ID: 9c9b618a9c754af99cf006cea78a6639
I20260809 03:29:53.165769 1185 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1: loading cluster ID for follower catalog manager: success
I20260809 03:29:53.167794 1185 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1: acquiring CA information for follower catalog manager: success
I20260809 03:29:53.169188 1185 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b17d4e94c9a1417489b3356be3fc8cf1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:29:53.189311 1206 catalog_manager.cc:1259] Loaded cluster ID: 9c9b618a9c754af99cf006cea78a6639
I20260809 03:29:53.189399 1206 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72: loading cluster ID for follower catalog manager: success
I20260809 03:29:53.191094 1206 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72: acquiring CA information for follower catalog manager: success
I20260809 03:29:53.192291 1206 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 703a4e08902545789dacd004d4738b72: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:29:53.381537 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:53.377409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53724 (local address 127.9.46.253:36051)
0809 03:29:53.377679 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:53.377686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:53.377773 (+ 87us) server_negotiation.cc:407] Connection header received
0809 03:29:53.377960 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:53.377964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:53.378106 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:53.378233 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:53.378522 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:53.379051 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:53.379955 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:53.380200 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:53.380317 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:53.380676 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:53.381112 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:53.381254 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:53.381316 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 03:29:53.381373 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":68,"threads_started":1}
I20260809 03:29:53.388621 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:53.384894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.9.46.252:35393)
0809 03:29:53.385160 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:53.385164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:53.385178 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:29:53.385332 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:53.385334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:53.385450 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:53.385564 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:53.385770 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:53.386312 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:53.387151 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:53.387377 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:53.387467 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:53.387734 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:53.388184 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:53.388279 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:53.388337 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 03:29:53.388386 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260809 03:29:53.395829 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:53.391821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.9.46.252:35393)
0809 03:29:53.391940 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:53.391944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:53.392233 (+ 289us) server_negotiation.cc:407] Connection header received
0809 03:29:53.392428 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:53.392431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:53.392474 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:53.392590 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:53.392820 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:53.393411 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:53.394568 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:53.394945 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:53.395042 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:53.395319 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:53.395406 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:53.395478 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:53.395611 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 03:29:53.395670 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 03:29:53.396288 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:53.391696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53740 (local address 127.9.46.253:36051)
0809 03:29:53.391956 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:53.391960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:53.392235 (+ 275us) server_negotiation.cc:407] Connection header received
0809 03:29:53.392471 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:53.392474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:53.392539 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:53.392640 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:53.392984 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:53.393793 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:53.394901 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:53.395220 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:53.395338 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:53.395750 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:53.395839 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:53.395887 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:53.396050 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 03:29:53.396139 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260809 03:29:53.397370 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:53.391889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.9.46.254:44325)
0809 03:29:53.392178 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:53.392184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:53.392200 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:29:53.392386 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:53.392390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:53.392575 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:53.392686 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:53.393065 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:53.393855 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:53.394827 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:53.395190 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:53.395297 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:53.396503 (+ 1206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:53.396905 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:53.397018 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:53.397115 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 03:29:53.397186 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":95,"threads_started":1}
I20260809 03:29:53.401408 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 914
I20260809 03:29:53.411625 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 994
I20260809 03:29:53.420421 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 1073
2026-08-09T03:29:53Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3182 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 03:29:53 dist-test-slave-65mx krb5kdc[1305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 03:29:53 dist-test-slave-65mx krb5kdc[1305](info): set up 2 sockets
Aug 09 03:29:53 dist-test-slave-65mx krb5kdc[1305](info): commencing operation
krb5kdc: starting...
W20260809 03:29:54.533382 9403 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/test-user.keytab.
Aug 09 03:29:54 dist-test-slave-65mx krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T03:29:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T03:29:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.46.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
Entry for principal kudu/127.9.46.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab.
I20260809 03:29:54.665869 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:37371
--webserver_interface=127.9.46.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:37371,127.9.46.253:41429,127.9.46.252:44357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 03:29:54.776029 1318 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:29:54.776364 1318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:54.776437 1318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:54.776520 1318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 03:29:54.780561 1318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:54.780651 1318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:54.780691 1318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:54.780731 1318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:54.780769 1318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:54.785943 1318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:37371,127.9.46.253:41429,127.9.46.252:44357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.254.keytab
--principal=kudu/127.9.46.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.254:37371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.46.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:54.786986 1318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:54.787976 1318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:54.796581 1324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:54.796670 1323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:54.796670 1326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:54.796859 1318 server_base.cc:1034] running on GCE node
I20260809 03:29:54.797536 1318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:54.798611 1318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:54.799793 1318 hybrid_clock.cc:648] HybridClock initialized: now 1786246194799781 us; error 28 us; skew 500 ppm
Aug 09 03:29:54 dist-test-slave-65mx krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246194, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:54.802765 1318 init.cc:377] Logged in from keytab as kudu/127.9.46.254@KRBTEST.COM (short username kudu)
I20260809 03:29:54.803675 1318 webserver.cc:460] Webserver started at http://127.9.46.254:38301/ using document root <none> and password file <none>
I20260809 03:29:54.804241 1318 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:54.804332 1318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:54.804618 1318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:54.806627 1318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/data/instance:
uuid: "301cb566c4ba454baf6da57c97a32855"
format_stamp: "Formatted at 2026-08-09 03:29:54 on dist-test-slave-65mx"
server_key: "8ff381540a33fa60f8161c314f56f16e"
server_key_iv: "d0acc60460fdf7566b73a20414d7911f"
server_key_version: "encryptionkey@0"
I20260809 03:29:54.807195 1318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/wal/instance:
uuid: "301cb566c4ba454baf6da57c97a32855"
format_stamp: "Formatted at 2026-08-09 03:29:54 on dist-test-slave-65mx"
server_key: "8ff381540a33fa60f8161c314f56f16e"
server_key_iv: "d0acc60460fdf7566b73a20414d7911f"
server_key_version: "encryptionkey@0"
I20260809 03:29:54.810845 1318 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 03:29:54.813365 1333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:54.814584 1318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:54.814730 1318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/wal
uuid: "301cb566c4ba454baf6da57c97a32855"
format_stamp: "Formatted at 2026-08-09 03:29:54 on dist-test-slave-65mx"
server_key: "8ff381540a33fa60f8161c314f56f16e"
server_key_iv: "d0acc60460fdf7566b73a20414d7911f"
server_key_version: "encryptionkey@0"
I20260809 03:29:54.814848 1318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:54.837853 1318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:54.841358 1318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:54.841601 1318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:54.849613 1318 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.254:37371
I20260809 03:29:54.849905 1387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.254:37371 every 8 connection(s)
I20260809 03:29:54.850703 1318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/data/info.pb
I20260809 03:29:54.852283 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 1318
I20260809 03:29:54.852391 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-0/wal/instance
I20260809 03:29:54.852758 9403 external_mini_cluster.cc:1428] Setting key a5d9ab7e2019d04ad23c361b657cdb44
I20260809 03:29:54.853636 1388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:54.859193 1388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.46.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.253@KRBTEST.COM" created.
Aug 09 03:29:54 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246194, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:54.876300 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:54.863030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.9.46.254:37371)
0809 03:29:54.863415 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:54.863434 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:54.863474 (+ 40us) server_negotiation.cc:407] Connection header received
0809 03:29:54.863982 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:54.863986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:54.864191 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:54.864382 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:54.865330 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:54.866745 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:54.867762 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:54.868410 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:54.868611 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:54.871353 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:54.871358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:54.871368 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:54.871388 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:54.873073 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:54.873573 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:54.873578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:54.873580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:54.873694 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:54.874009 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:54.874012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:54.874014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:54.874382 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:54.874523 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:54.874920 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 03:29:54.875239 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":237,"thread_start_us":107,"threads_started":1}
I20260809 03:29:54.876300 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:54.862107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37371 (local address 127.0.0.1:34562)
0809 03:29:54.862630 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:54.862685 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:54.863701 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:54.864406 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:54.864411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:54.864822 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:54.865182 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:54.865197 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:54.866890 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:54.866893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:54.867634 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:54.867643 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:54.868602 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:54.868609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:54.868765 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:54.869385 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:54.869416 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:54.871196 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:54.873203 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:54.873213 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:54.873222 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:54.873448 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:54.873794 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:54.873797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:54.873799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:54.873912 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:54.874531 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:54.874540 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:54.874673 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 03:29:54.874840 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 03:29:54.875288 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":165,"thread_start_us":161,"threads_started":1}
I20260809 03:29:54.878805 1388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:54.879606 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:54.879315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:41429 (local address 127.0.0.1:43092)
0809 03:29:54.879464 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:54.879539 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:41429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 03:29:54.879842 1337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.46.253:41429: connect: Connection refused (error 111)
I20260809 03:29:54.880798 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:54.880577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:41429 (local address 127.0.0.1:43102)
0809 03:29:54.880696 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:54.880741 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:41429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 03:29:54.880959 1388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.253:41429: Network error: Client connection negotiation failed: client connection to 127.9.46.253:41429: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab.
Entry for principal kudu/127.9.46.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab.
I20260809 03:29:54.884114 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.253:41429
--webserver_interface=127.9.46.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab
--principal=kudu/127.9.46.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:37371,127.9.46.253:41429,127.9.46.252:44357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:29:54.930228 1388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } attempt: 1
I20260809 03:29:54.931239 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:54.930866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:41429 (local address 127.0.0.1:43108)
0809 03:29:54.931049 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:54.931121 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:41429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 03:29:54.932262 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:54.932012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:41429 (local address 127.0.0.1:43120)
0809 03:29:54.932150 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:54.932207 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:41429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 03:29:54.932475 1388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.253:41429: Network error: Client connection negotiation failed: client connection to 127.9.46.253:41429: connect: Connection refused (error 111)
W20260809 03:29:55.000347 1395 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:29:55.000695 1395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:55.000741 1395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260809 03:29:55.000679 1388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } attempt: 2
W20260809 03:29:55.000802 1395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 03:29:55.001467 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.001179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:41429 (local address 127.0.0.1:43124)
0809 03:29:55.001321 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.001393 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:41429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 03:29:55.002230 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.002007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:41429 (local address 127.0.0.1:43136)
0809 03:29:55.002113 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.002170 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.253:41429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 03:29:55.002364 1388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.253:41429: Network error: Client connection negotiation failed: client connection to 127.9.46.253:41429: connect: Connection refused (error 111)
W20260809 03:29:55.005057 1395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:55.005105 1395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:55.005124 1395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:55.005177 1395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:55.005230 1395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:55.010480 1395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:37371,127.9.46.253:41429,127.9.46.252:44357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.253.keytab
--principal=kudu/127.9.46.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.253:41429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.46.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:55.011459 1395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:55.012439 1395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:55.020836 1405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:55.020843 1401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:55.020843 1403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:55.021139 1395 server_base.cc:1034] running on GCE node
I20260809 03:29:55.021742 1395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:55.022807 1395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:55.023998 1395 hybrid_clock.cc:648] HybridClock initialized: now 1786246195023969 us; error 40 us; skew 500 ppm
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246195, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:55.026764 1395 init.cc:377] Logged in from keytab as kudu/127.9.46.253@KRBTEST.COM (short username kudu)
I20260809 03:29:55.027608 1395 webserver.cc:460] Webserver started at http://127.9.46.253:38679/ using document root <none> and password file <none>
I20260809 03:29:55.028149 1395 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:55.028239 1395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:55.028463 1395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:55.030546 1395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/data/instance:
uuid: "8814186be8e24fbcb2e72d6692e5c344"
format_stamp: "Formatted at 2026-08-09 03:29:55 on dist-test-slave-65mx"
server_key: "d6a058b3095af68e90450a729eee1fd1"
server_key_iv: "4d4d56e6f218336180a36de8c966f333"
server_key_version: "encryptionkey@0"
I20260809 03:29:55.031111 1395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/wal/instance:
uuid: "8814186be8e24fbcb2e72d6692e5c344"
format_stamp: "Formatted at 2026-08-09 03:29:55 on dist-test-slave-65mx"
server_key: "d6a058b3095af68e90450a729eee1fd1"
server_key_iv: "4d4d56e6f218336180a36de8c966f333"
server_key_version: "encryptionkey@0"
I20260809 03:29:55.034686 1395 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:29:55.036963 1412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:55.038115 1395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:55.038249 1395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/wal
uuid: "8814186be8e24fbcb2e72d6692e5c344"
format_stamp: "Formatted at 2026-08-09 03:29:55 on dist-test-slave-65mx"
server_key: "d6a058b3095af68e90450a729eee1fd1"
server_key_iv: "4d4d56e6f218336180a36de8c966f333"
server_key_version: "encryptionkey@0"
I20260809 03:29:55.038355 1395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:55.050364 1395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:55.053089 1395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:55.053280 1395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:55.060125 1395 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.253:41429
I20260809 03:29:55.060412 1468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.253:41429 every 8 connection(s)
I20260809 03:29:55.061156 1395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/data/info.pb
I20260809 03:29:55.063858 1469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:55.067451 1469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:55.071017 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 1395
I20260809 03:29:55.071205 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-1/wal/instance
I20260809 03:29:55.071487 9403 external_mini_cluster.cc:1428] Setting key fc8a72992370dca4ba6f2058b4c435fb
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246195, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.46.252@KRBTEST.COM; defaulting to no policy
I20260809 03:29:55.081760 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.069337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34578 (local address 127.9.46.254:37371)
0809 03:29:55.069479 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.069483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.070282 (+ 799us) server_negotiation.cc:407] Connection header received
0809 03:29:55.071202 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.071206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.071268 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.071351 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.072493 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.073083 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.074199 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.074434 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.074549 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.077489 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.077492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.077494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.077520 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.079528 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.080152 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.080155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.080156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.080232 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.080539 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.080579 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.080581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.080803 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.080914 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.081417 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.081542 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.46.252@KRBTEST.COM" created.
I20260809 03:29:55.082592 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.069676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37371 (local address 127.0.0.1:34578)
0809 03:29:55.070131 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.070172 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:55.070989 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:55.071508 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:55.071514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:55.071895 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:55.072342 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.072354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.073230 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.073233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.074064 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.074074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.074587 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.074591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.074763 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.075366 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:55.075382 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:55.077311 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:55.079652 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.079658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.079672 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.080050 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.080320 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.080322 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.080323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.080457 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.080915 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:55.080923 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:55.081172 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 03:29:55.081360 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 03:29:55.081667 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":128,"threads_started":1}
I20260809 03:29:55.084061 1469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246195, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
I20260809 03:29:55.093587 1388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } attempt: 3
I20260809 03:29:55.094565 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.084610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:41429 (local address 127.0.0.1:43152)
0809 03:29:55.084717 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.084734 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:55.084809 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:55.085584 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:55.085587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:55.085731 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:55.085904 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.085915 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.087075 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.087079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.088055 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.088069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.088819 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.088827 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.088957 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.089410 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:55.089428 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:55.090770 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:55.092839 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.092842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.092844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.093033 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.093378 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.093381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.093383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.093465 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.094047 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:55.094052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:55.094150 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 03:29:55.094260 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 03:29:55.094403 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":47}
I20260809 03:29:55.094682 1474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.084783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43152 (local address 127.9.46.253:41429)
0809 03:29:55.085142 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.085155 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.085179 (+ 24us) server_negotiation.cc:407] Connection header received
0809 03:29:55.085224 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.085232 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.085426 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.085577 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.086187 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.086939 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.088200 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.088690 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.088786 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.090898 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.090916 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.090927 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.090946 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.092712 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.093171 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.093178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.093183 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.093271 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.093567 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.093570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.093572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.093891 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.094030 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.094390 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.094511 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":248,"thread_start_us":140,"threads_started":1}
I20260809 03:29:55.097018 1469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:55.097707 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.097464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33524)
0809 03:29:55.097572 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.097653 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 03:29:55.098373 1416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.46.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab.
Entry for principal kudu/127.9.46.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab.
I20260809 03:29:55.099277 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.099063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33532)
0809 03:29:55.099149 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.099196 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246194, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
W20260809 03:29:55.099436 1469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:44357: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
I20260809 03:29:55.100234 9403 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
/tmp/dist-test-taskOU4Lix/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.252:44357
--webserver_interface=127.9.46.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab
--principal=kudu/127.9.46.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.46.212:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.46.254:37371,127.9.46.253:41429,127.9.46.252:44357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 03:29:55.103192 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.094038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:41429 (local address 127.0.0.1:43164)
0809 03:29:55.094187 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.094204 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:55.094279 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:55.094802 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:55.094807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:55.094984 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:55.095123 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.095130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.096028 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.096032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.096869 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.096879 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.097758 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.097761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.097867 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.098356 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:55.098367 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:55.099872 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:55.101699 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.101702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.101705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.101908 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.102201 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.102205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.102208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.102301 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.102671 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:55.102674 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:55.102779 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 03:29:55.102888 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 03:29:55.103033 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 03:29:55.103209 1475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.094075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.9.46.253:41429)
0809 03:29:55.094528 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.094536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.094548 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:29:55.094594 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.094597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.094657 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.094758 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.095247 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.095897 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.097362 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.097635 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.097754 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.100019 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.100022 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.100024 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.100051 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.101563 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.102009 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.102013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.102015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.102106 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.102393 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.102397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.102399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.102573 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.102683 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.102940 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.103043 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":197,"threads_started":1}
I20260809 03:29:55.103889 1388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:55.104756 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.104404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33536)
0809 03:29:55.104598 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.104666 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 03:29:55.105579 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.105256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33552)
0809 03:29:55.105429 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.105492 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260809 03:29:55.105718 1388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:44357: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
I20260809 03:29:55.136900 1388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } attempt: 1
I20260809 03:29:55.137820 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.137475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33568)
0809 03:29:55.137655 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.137727 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 03:29:55.138543 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.138327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33582)
0809 03:29:55.138434 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.138487 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 03:29:55.138701 1388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:44357: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
I20260809 03:29:55.148645 1469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } attempt: 1
I20260809 03:29:55.149376 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.149117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33594)
0809 03:29:55.149247 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.149309 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 03:29:55.150166 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.149959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33602)
0809 03:29:55.150055 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.150120 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 03:29:55.150326 1469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:44357: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
W20260809 03:29:55.212064 1477 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 03:29:55.212327 1477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 03:29:55.212350 1477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 03:29:55.212387 1477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 03:29:55.213922 1388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } attempt: 2
I20260809 03:29:55.214852 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.214495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33616)
0809 03:29:55.214689 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.214762 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 03:29:55.215601 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.215386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33620)
0809 03:29:55.215485 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.215544 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 03:29:55.215728 1388 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:44357: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
W20260809 03:29:55.217080 1477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 03:29:55.217141 1477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 03:29:55.217168 1477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 03:29:55.217199 1477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 03:29:55.217221 1477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 03:29:55.218470 1469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } attempt: 2
I20260809 03:29:55.219153 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.218901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33634)
0809 03:29:55.219034 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.219084 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 03:29:55.219832 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.219647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33646)
0809 03:29:55.219743 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.219786 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 03:29:55.219956 1469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.46.252:44357: Network error: Client connection negotiation failed: client connection to 127.9.46.252:44357: connect: Connection refused (error 111)
I20260809 03:29:55.223070 1477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.46.212:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.46.254:37371,127.9.46.253:41429,127.9.46.252:44357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/krb5kdc/kudu_127.9.46.252.keytab
--principal=kudu/127.9.46.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.46.252:44357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.46.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 02:17:35 UTC on ccycloud.quasar-qimiwd.root.comops.site
build id 4575723
I20260809 03:29:55.224138 1477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 03:29:55.225172 1477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 03:29:55.233194 1486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 03:29:55.233212 1484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:55.233341 1477 server_base.cc:1034] running on GCE node
W20260809 03:29:55.233225 1483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 03:29:55.234036 1477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 03:29:55.235090 1477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 03:29:55.236277 1477 hybrid_clock.cc:648] HybridClock initialized: now 1786246195236261 us; error 30 us; skew 500 ppm
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246195, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 03:29:55.239060 1477 init.cc:377] Logged in from keytab as kudu/127.9.46.252@KRBTEST.COM (short username kudu)
I20260809 03:29:55.239914 1477 webserver.cc:460] Webserver started at http://127.9.46.252:44387/ using document root <none> and password file <none>
I20260809 03:29:55.240415 1477 fs_manager.cc:359] Metadata directory not provided
I20260809 03:29:55.240473 1477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 03:29:55.240715 1477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 03:29:55.242627 1477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/data/instance:
uuid: "45c2e9941b904ed187bbe215f138a18e"
format_stamp: "Formatted at 2026-08-09 03:29:55 on dist-test-slave-65mx"
server_key: "847bca2d9810782f42cbafa9531130ce"
server_key_iv: "7bf0554e6cf07559ed655547f01ac7eb"
server_key_version: "encryptionkey@0"
I20260809 03:29:55.243121 1477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/wal/instance:
uuid: "45c2e9941b904ed187bbe215f138a18e"
format_stamp: "Formatted at 2026-08-09 03:29:55 on dist-test-slave-65mx"
server_key: "847bca2d9810782f42cbafa9531130ce"
server_key_iv: "7bf0554e6cf07559ed655547f01ac7eb"
server_key_version: "encryptionkey@0"
I20260809 03:29:55.246661 1477 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 03:29:55.249011 1493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 03:29:55.250213 1477 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 03:29:55.250340 1477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/data,/tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/wal
uuid: "45c2e9941b904ed187bbe215f138a18e"
format_stamp: "Formatted at 2026-08-09 03:29:55 on dist-test-slave-65mx"
server_key: "847bca2d9810782f42cbafa9531130ce"
server_key_iv: "7bf0554e6cf07559ed655547f01ac7eb"
server_key_version: "encryptionkey@0"
I20260809 03:29:55.250445 1477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 03:29:55.267315 1477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 03:29:55.270262 1477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 03:29:55.270463 1477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 03:29:55.277716 1477 rpc_server.cc:307] RPC server started. Bound to: 127.9.46.252:44357
I20260809 03:29:55.278000 1552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.46.252:44357 every 8 connection(s)
I20260809 03:29:55.278688 1477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/data/info.pb
I20260809 03:29:55.281354 1553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 03:29:55.284919 1553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:55.287066 9403 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu as pid 1477
I20260809 03:29:55.287249 9403 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOU4Lix/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786245904130257-9403-0/minicluster-data/master-2/wal/instance
I20260809 03:29:55.287567 9403 external_mini_cluster.cc:1428] Setting key ae51e007b23a520568e18583793b1ae4
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246195, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
I20260809 03:29:55.299415 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.286654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34588 (local address 127.9.46.254:37371)
0809 03:29:55.286788 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.286791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.287554 (+ 763us) server_negotiation.cc:407] Connection header received
0809 03:29:55.288829 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.288833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.288899 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.289002 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.290139 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.290955 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.292278 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.292573 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.292679 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.295408 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.295414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.295422 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.295449 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.297258 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.297843 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.297846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.297849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.297946 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.298280 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.298285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.298287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.298491 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.298623 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.299115 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.299240 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 03:29:55.300328 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.286928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.254:37371 (local address 127.0.0.1:34588)
0809 03:29:55.287384 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.287436 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:55.288597 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:55.289089 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:55.289097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:55.289547 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:55.289959 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.289978 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.291084 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.291087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.292158 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.292168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.292719 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.292722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.292896 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.293488 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:55.293532 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:55.295256 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:55.297380 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.297388 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.297396 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.297740 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.298027 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.298030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.298032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.298180 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.298623 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:55.298632 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:55.298879 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 03:29:55.299078 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 03:29:55.299385 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":19,"thread_start_us":147,"threads_started":1}
I20260809 03:29:55.300333 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.288243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33656 (local address 127.9.46.252:44357)
0809 03:29:55.288494 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.288504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.288781 (+ 277us) server_negotiation.cc:407] Connection header received
0809 03:29:55.288971 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.288978 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.289195 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.289329 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.290227 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.291356 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.292458 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.292988 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.293066 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.295880 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.295886 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.295915 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.295942 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.297845 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.298338 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.298344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.298346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.298435 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.298744 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.298747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.298749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.298990 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.299251 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.299405 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.299788 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260809 03:29:55.301920 1553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246195, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.254@KRBTEST.COM
I20260809 03:29:55.311086 1475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.302582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43166 (local address 127.9.46.253:41429)
0809 03:29:55.302715 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.302720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.302741 (+ 21us) server_negotiation.cc:407] Connection header received
0809 03:29:55.302801 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.302804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.302855 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.302940 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.303472 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.304155 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.305122 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.305368 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.305491 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.307557 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.307560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.307562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.307587 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.309448 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.309871 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.309876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.309878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.309969 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.310270 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.310275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.310276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.310456 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.310567 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.310830 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.310946 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 03:29:55.311067 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.302451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.253:41429 (local address 127.0.0.1:43166)
0809 03:29:55.302635 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.302651 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:55.302722 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:55.303065 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:55.303068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:55.303207 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:55.303339 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.303350 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.304269 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.304271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.305005 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.305015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.305509 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.305512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.305586 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.306052 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:55.306085 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:55.307415 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:55.309562 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.309564 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.309566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.309766 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.310071 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.310073 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.310075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.310153 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.310558 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:55.310561 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:55.310667 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 03:29:55.310777 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 03:29:55.310904 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 03:29:55.311136 1469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } attempt: 3
I20260809 03:29:55.311931 1553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } has no permanent_uuid. Determining permanent_uuid...
I20260809 03:29:55.312779 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.302995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34600 (local address 127.9.46.254:37371)
0809 03:29:55.303101 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.303105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.303124 (+ 19us) server_negotiation.cc:407] Connection header received
0809 03:29:55.303151 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.303153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.303184 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.303243 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.303885 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.304520 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.305451 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.305719 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.305831 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.308431 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.308434 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.308437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.308462 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.310129 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.310534 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.310537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.310539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.310630 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.310906 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.310910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.310911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.311124 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.311754 (+ 630us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.312455 (+ 701us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.312621 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:29:55.314872 1388 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } attempt: 3
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246195, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.253@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.46.253@KRBTEST.COM
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246195, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.252@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
Aug 09 03:29:55 dist-test-slave-65mx krb5kdc[1305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786246194, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.46.254@KRBTEST.COM for kudu/127.9.46.252@KRBTEST.COM
I20260809 03:29:55.322182 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.311545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33670)
0809 03:29:55.311835 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.311848 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:55.311950 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:55.312195 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:55.312198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:55.312381 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:55.312492 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.312499 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.313426 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.313429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.314172 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.314182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.314897 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.314900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.314979 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.315481 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:55.315495 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:55.317189 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:55.320447 (+ 3258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.320452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.320457 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.320741 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.321086 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.321088 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.321089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.321176 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.321644 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:55.321650 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:55.321757 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 03:29:55.321849 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 03:29:55.321991 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221}
I20260809 03:29:55.322821 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.311693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33670 (local address 127.9.46.252:44357)
0809 03:29:55.311811 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.311815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.311913 (+ 98us) server_negotiation.cc:407] Connection header received
0809 03:29:55.312051 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.312053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.312098 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.312178 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.312635 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.313281 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.314340 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.314776 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.314897 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.317326 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.317329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.317332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.317355 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.320321 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.320874 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.320878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.320880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.320966 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.321295 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.321298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.321300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.321521 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.322440 (+ 919us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.322517 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.322634 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":44}
I20260809 03:29:55.325650 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.312904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33676)
0809 03:29:55.312994 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.313007 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:55.313071 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:55.313964 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:55.313966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:55.314140 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:55.314229 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.314236 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.316195 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.316197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.317136 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.317147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.317738 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.317743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.317840 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.318304 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:55.318320 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:55.322198 (+ 3878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:55.324215 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.324221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.324224 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.324415 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.324726 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.324729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.324731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.324812 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.325164 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:55.325167 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:55.325247 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 03:29:55.325356 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 03:29:55.325491 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 03:29:55.325701 1558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.313372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.9.46.252:44357)
0809 03:29:55.313691 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.313702 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.313718 (+ 16us) server_negotiation.cc:407] Connection header received
0809 03:29:55.313752 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.313755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.313799 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.313877 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.314344 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.316091 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.317258 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.317617 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.317723 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.322336 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.322339 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.322342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.322365 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.324108 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.324509 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.324514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.324516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.324620 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.324905 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.324908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.324910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.325067 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.325172 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.325411 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.325543 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":240,"thread_start_us":110,"threads_started":1}
I20260809 03:29:55.326371 1475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.314332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.9.46.253:41429)
0809 03:29:55.314433 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.314437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.314515 (+ 78us) server_negotiation.cc:407] Connection header received
0809 03:29:55.314636 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.314638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.314677 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.314756 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.315567 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.316257 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.317349 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.317652 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.317737 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.321978 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.321983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.321986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.322013 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.324510 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.325023 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.325027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.325028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.325121 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.325431 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.325438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.325442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.325647 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.325818 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.326141 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.326227 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 03:29:55.326538 1469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344: Bootstrap starting.
I20260809 03:29:55.329591 1469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:55.329668 9403 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 03:29:55.330533 1469 log.cc:824] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:55.331139 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.315284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.46.252:44357 (local address 127.0.0.1:33680)
0809 03:29:55.315530 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 03:29:55.315545 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 03:29:55.315619 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 03:29:55.316487 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 03:29:55.316490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 03:29:55.316702 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 03:29:55.316830 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.316841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.318280 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.318287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.318957 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 03:29:55.318966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.319413 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.319419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 03:29:55.319523 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.320000 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 03:29:55.320016 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 03:29:55.321693 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 03:29:55.329510 (+ 7817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.329513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.329515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.329791 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.330114 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 03:29:55.330117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 03:29:55.330119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 03:29:55.330216 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 03:29:55.330583 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 03:29:55.330586 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 03:29:55.330690 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 03:29:55.330819 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 03:29:55.330968 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260809 03:29:55.331157 1559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.315778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33680 (local address 127.9.46.252:44357)
0809 03:29:55.316244 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.316252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.316265 (+ 13us) server_negotiation.cc:407] Connection header received
0809 03:29:55.316296 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.316299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.316335 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.316974 (+ 639us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.317106 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.318145 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.319077 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.319299 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.319366 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.321828 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.321831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.321833 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.321856 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.329401 (+ 7545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.329905 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.329909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.329911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.329999 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.330317 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.330320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.330322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.330486 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.330588 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.330871 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.330982 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":103,"threads_started":1}
I20260809 03:29:55.330956 1361 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "301cb566c4ba454baf6da57c97a32855"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34600
I20260809 03:29:55.333316 1553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e: Bootstrap starting.
I20260809 03:29:55.334664 1388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855: Bootstrap starting.
I20260809 03:29:55.335935 1553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:55.336864 1553 log.cc:824] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:55.337102 1388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855: Neither blocks nor log segments found. Creating new log.
I20260809 03:29:55.337937 1388 log.cc:824] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855: Log is configured to *not* fsync() on all Append() calls
I20260809 03:29:55.339133 1553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e: No bootstrap required, opened a new log
I20260809 03:29:55.339825 1388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855: No bootstrap required, opened a new log
I20260809 03:29:55.341841 1469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344: No bootstrap required, opened a new log
I20260809 03:29:55.342734 1388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } }
I20260809 03:29:55.342962 1388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301cb566c4ba454baf6da57c97a32855, State: Initialized, Role: FOLLOWER
I20260809 03:29:55.343310 1388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } }
I20260809 03:29:55.343817 1563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } } }
I20260809 03:29:55.343958 1563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:55.344750 1388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:55.344827 1361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 03:29:55.344950 1361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:55.345834 1469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } }
I20260809 03:29:55.346081 1469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8814186be8e24fbcb2e72d6692e5c344, State: Initialized, Role: FOLLOWER
I20260809 03:29:55.346159 1361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } }
I20260809 03:29:55.346431 1469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } }
I20260809 03:29:55.347891 1469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 03:29:55.347963 1442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "301cb566c4ba454baf6da57c97a32855" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8814186be8e24fbcb2e72d6692e5c344"
I20260809 03:29:55.348198 1442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:55.349624 1442 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301cb566c4ba454baf6da57c97a32855 in term 1.
I20260809 03:29:55.350221 1337 leader_election.cc:302] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301cb566c4ba454baf6da57c97a32855, 8814186be8e24fbcb2e72d6692e5c344; no voters:
I20260809 03:29:55.350497 1563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 03:29:55.350787 1563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [term 1 LEADER]: Becoming Leader. State: Replica: 301cb566c4ba454baf6da57c97a32855, State: Running, Role: LEADER
I20260809 03:29:55.351118 1563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } }
I20260809 03:29:55.352974 1563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 301cb566c4ba454baf6da57c97a32855. Latest consensus state: current_term: 1 leader_uuid: "301cb566c4ba454baf6da57c97a32855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } } }
I20260809 03:29:55.353106 1563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:55.350499 1318 master_runner.cc:186] Error getting master registration for 127.9.46.253:41429: OK, instance_id { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" instance_seqno: 1786246195058327 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:29:55.354650 1318 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 03:29:55.347219 1361 leader_election.cc:288] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [CANDIDATE]: Term 1 election: Requested vote from peers 8814186be8e24fbcb2e72d6692e5c344 (127.9.46.253:41429), 45c2e9941b904ed187bbe215f138a18e (127.9.46.252:44357)
I20260809 03:29:55.355057 1553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } }
I20260809 03:29:55.355377 1553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c2e9941b904ed187bbe215f138a18e, State: Initialized, Role: FOLLOWER
I20260809 03:29:55.355703 1553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } }
W20260809 03:29:55.356356 1591 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:29:55.356515 1591 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:29:55.357471 1553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [sys.catalog]: configured and running, proceeding with master startup.
W20260809 03:29:55.358214 1575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:29:55.358282 1575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:29:55.358261 1524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "301cb566c4ba454baf6da57c97a32855" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "45c2e9941b904ed187bbe215f138a18e"
I20260809 03:29:55.358485 1524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [term 0 FOLLOWER]: Advancing to term 1
I20260809 03:29:55.358541 1595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 03:29:55.358839 1573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } } }
I20260809 03:29:55.358948 1573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:55.359443 1595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 03:29:55.359942 1524 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301cb566c4ba454baf6da57c97a32855 in term 1.
I20260809 03:29:55.362061 1600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } } }
I20260809 03:29:55.362197 1600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:55.360917 1395 master_runner.cc:186] Error getting master registration for 127.9.46.252:44357: OK, instance_id { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" instance_seqno: 1786246195275645 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 03:29:55.364506 1395 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 03:29:55.366112 1614 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 03:29:55.366235 1614 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 03:29:55.369042 1524 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [term 1 FOLLOWER]: Refusing update from remote peer 301cb566c4ba454baf6da57c97a32855: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:55.369742 1442 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [term 1 FOLLOWER]: Refusing update from remote peer 301cb566c4ba454baf6da57c97a32855: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 03:29:55.370525 1563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:55.370957 1563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 03:29:55.374763 1573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 301cb566c4ba454baf6da57c97a32855. Latest consensus state: current_term: 1 leader_uuid: "301cb566c4ba454baf6da57c97a32855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } } }
I20260809 03:29:55.374881 1573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:55.376943 1475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.360721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.9.46.253:41429)
0809 03:29:55.360820 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.360825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.365790 (+ 4965us) server_negotiation.cc:407] Connection header received
0809 03:29:55.365960 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.365964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.366011 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.366059 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.368377 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.369142 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.370657 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.370974 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.371047 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.373014 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.373021 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.373026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.373055 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.374839 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.375400 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.375405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.375408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.375505 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.375867 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.375871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.375875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.376130 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.376245 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.376682 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.376800 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 03:29:55.378615 1600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 301cb566c4ba454baf6da57c97a32855. Latest consensus state: current_term: 1 leader_uuid: "301cb566c4ba454baf6da57c97a32855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } } }
I20260809 03:29:55.378741 1600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:55.380671 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.356924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.9.46.254:37371)
0809 03:29:55.357056 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.357060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.360091 (+ 3031us) server_negotiation.cc:407] Connection header received
0809 03:29:55.360380 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.360383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.360434 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.360489 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.362344 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.363112 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.364279 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.364637 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.364715 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.374949 (+ 10234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.374956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.374959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.374986 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.376759 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.377806 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.377813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.377817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.377910 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.379646 (+ 1736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.379650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.379652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.379888 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.379990 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.380346 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.380468 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 03:29:55.381435 1559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:55.357741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.9.46.252:44357)
0809 03:29:55.357862 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.357866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.367257 (+ 9391us) server_negotiation.cc:407] Connection header received
0809 03:29:55.367420 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.367423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.367469 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.367518 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.368331 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.369168 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.370613 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:55.370970 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:55.371062 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:55.377456 (+ 6394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 03:29:55.377459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 03:29:55.377462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 03:29:55.377489 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 03:29:55.379332 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.379815 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.379818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.379820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.379901 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 03:29:55.380217 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 03:29:55.380223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 03:29:55.380228 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 03:29:55.380444 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 03:29:55.380590 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:55.381177 (+ 587us) server_negotiation.cc:299] Negotiation successful
0809 03:29:55.381301 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 03:29:55.389756 1592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "301cb566c4ba454baf6da57c97a32855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } } }
I20260809 03:29:55.389889 1592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:55.390173 1592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "301cb566c4ba454baf6da57c97a32855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } } }
I20260809 03:29:55.390266 1592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855 [sys.catalog]: This master's current role is: LEADER
I20260809 03:29:55.390610 1617 mvcc.cc:205] Tried to move back new op lower bound from 7316464416223903744 to 7316464416161640448. Current Snapshot: MvccSnapshot[applied={T|T < 7316464416223903744}]
I20260809 03:29:55.391201 1600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "301cb566c4ba454baf6da57c97a32855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } } }
I20260809 03:29:55.391233 1573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "301cb566c4ba454baf6da57c97a32855" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "301cb566c4ba454baf6da57c97a32855" member_type: VOTER last_known_addr { host: "127.9.46.254" port: 37371 } } peers { permanent_uuid: "8814186be8e24fbcb2e72d6692e5c344" member_type: VOTER last_known_addr { host: "127.9.46.253" port: 41429 } } peers { permanent_uuid: "45c2e9941b904ed187bbe215f138a18e" member_type: VOTER last_known_addr { host: "127.9.46.252" port: 44357 } } }
I20260809 03:29:55.391319 1600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:55.391350 1573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 03:29:55.391610 1621 mvcc.cc:205] Tried to move back new op lower bound from 7316464416223903744 to 7316464416161640448. Current Snapshot: MvccSnapshot[applied={T|T < 7316464416223903744}]
I20260809 03:29:55.391916 1619 mvcc.cc:205] Tried to move back new op lower bound from 7316464416223903744 to 7316464416161640448. Current Snapshot: MvccSnapshot[applied={T|T < 7316464416223903744}]
I20260809 03:29:55.404744 1595 catalog_manager.cc:1347] Generated new cluster ID: ede65754ce8e4e0589d29957a3c81344
I20260809 03:29:55.404825 1595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 03:29:55.418834 1595 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 03:29:55.420717 1595 catalog_manager.cc:1504] Loading token signing keys...
I20260809 03:29:55.457674 1595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 301cb566c4ba454baf6da57c97a32855: Generated new TSK 0
I20260809 03:29:55.458621 1595 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 03:29:55.492789 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.490292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.9.46.253:41429)
0809 03:29:55.490421 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.490426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.491696 (+ 1270us) server_negotiation.cc:407] Connection header received
0809 03:29:55.491894 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.491897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.491946 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.491998 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.492649 (+ 651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60504: BlockingRecv error: recv got EOF from 127.0.0.1:60504 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 03:29:55.494215 1559 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.491071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.9.46.252:44357)
0809 03:29:55.491181 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.491185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.493270 (+ 2085us) server_negotiation.cc:407] Connection header received
0809 03:29:55.493507 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.493510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.493559 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.493608 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.494099 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55492: BlockingRecv error: recv got EOF from 127.0.0.1:55492 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 03:29:55.496336 1392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.495067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.9.46.254:37371)
0809 03:29:55.495183 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.495187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.495468 (+ 281us) server_negotiation.cc:407] Connection header received
0809 03:29:55.495639 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.495643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.495691 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.495741 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.496221 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56042: BlockingRecv error: recv got EOF from 127.0.0.1:56042 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 03:29:55.500604 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.499081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60508 (local address 127.9.46.253:41429)
0809 03:29:55.499180 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.499184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.499587 (+ 403us) server_negotiation.cc:407] Connection header received
0809 03:29:55.499815 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.499818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.499897 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.499947 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.500461 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60508: BlockingRecv error: recv got EOF from 127.0.0.1:60508 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 03:29:55.505650 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.504152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.9.46.253:41429)
0809 03:29:55.504250 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.504254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.504771 (+ 517us) server_negotiation.cc:407] Connection header received
0809 03:29:55.504953 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.504956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.505007 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.505057 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.505536 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60514: BlockingRecv error: recv got EOF from 127.0.0.1:60514 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 03:29:55.511384 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.509902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60524 (local address 127.9.46.253:41429)
0809 03:29:55.509997 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.510001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.510480 (+ 479us) server_negotiation.cc:407] Connection header received
0809 03:29:55.510683 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.510686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.510738 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.510790 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.511289 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60524: BlockingRecv error: recv got EOF from 127.0.0.1:60524 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 03:29:55.519521 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.517667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.9.46.253:41429)
0809 03:29:55.517797 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.517801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.518416 (+ 615us) server_negotiation.cc:407] Connection header received
0809 03:29:55.518677 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.518680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.518734 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.518786 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.519410 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60538: BlockingRecv error: recv got EOF from 127.0.0.1:60538 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 03:29:55.531708 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.529893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.9.46.253:41429)
0809 03:29:55.530022 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.530027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.530651 (+ 624us) server_negotiation.cc:407] Connection header received
0809 03:29:55.530894 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.530897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.530955 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.531009 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.531585 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60546: BlockingRecv error: recv got EOF from 127.0.0.1:60546 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 03:29:55.555203 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.553087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60548 (local address 127.9.46.253:41429)
0809 03:29:55.553214 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.553219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.553946 (+ 727us) server_negotiation.cc:407] Connection header received
0809 03:29:55.554215 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.554218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.554280 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.554351 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.555076 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60548: BlockingRecv error: recv got EOF from 127.0.0.1:60548 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 03:29:55.591274 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.589913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.9.46.253:41429)
0809 03:29:55.590047 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.590052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.590323 (+ 271us) server_negotiation.cc:407] Connection header received
0809 03:29:55.590497 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.590500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.590560 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.590654 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.591153 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60558: BlockingRecv error: recv got EOF from 127.0.0.1:60558 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 03:29:55.658601 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.657337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.9.46.253:41429)
0809 03:29:55.657514 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.657518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.657686 (+ 168us) server_negotiation.cc:407] Connection header received
0809 03:29:55.657811 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.657814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.657885 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.657983 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.658453 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60568: BlockingRecv error: recv got EOF from 127.0.0.1:60568 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 03:29:55.789930 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:55.788755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60584 (local address 127.9.46.253:41429)
0809 03:29:55.789054 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:55.789057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:55.789092 (+ 35us) server_negotiation.cc:407] Connection header received
0809 03:29:55.789152 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:55.789155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:55.789203 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:55.789278 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:55.789775 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60584: BlockingRecv error: recv got EOF from 127.0.0.1:60584 (error 108)
Metrics: {"server-negotiator.queue_time_us":190}
W20260809 03:29:56.049365 1475 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 03:29:56.048052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.9.46.253:41429)
0809 03:29:56.048226 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:56.048230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:56.048324 (+ 94us) server_negotiation.cc:407] Connection header received
0809 03:29:56.048461 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:56.048464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:56.048514 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:56.048634 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 03:29:56.049233 (+ 599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60596: BlockingRecv error: recv got EOF from 127.0.0.1:60596 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 03:29:56.357985 1591 catalog_manager.cc:1259] Loaded cluster ID: ede65754ce8e4e0589d29957a3c81344
I20260809 03:29:56.358068 1591 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344: loading cluster ID for follower catalog manager: success
I20260809 03:29:56.359988 1591 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344: acquiring CA information for follower catalog manager: success
I20260809 03:29:56.361184 1591 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8814186be8e24fbcb2e72d6692e5c344: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:29:56.367355 1614 catalog_manager.cc:1259] Loaded cluster ID: ede65754ce8e4e0589d29957a3c81344
I20260809 03:29:56.367417 1614 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e: loading cluster ID for follower catalog manager: success
I20260809 03:29:56.368988 1614 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e: acquiring CA information for follower catalog manager: success
I20260809 03:29:56.370097 1614 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 45c2e9941b904ed187bbe215f138a18e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 03:29:56.567731 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:56.563714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60598 (local address 127.9.46.253:41429)
0809 03:29:56.563963 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:56.563967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:56.564030 (+ 63us) server_negotiation.cc:407] Connection header received
0809 03:29:56.564147 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:56.564149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:56.564278 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:56.564390 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:56.564688 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:56.565306 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:56.566340 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:56.566564 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:56.566658 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:56.566935 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:56.567372 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:56.567467 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:56.567543 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 03:29:56.567606 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
I20260809 03:29:56.574517 1707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:56.570829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.9.46.252:44357)
0809 03:29:56.571139 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:56.571143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:56.571157 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:29:56.571235 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:56.571237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:56.571355 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:56.571454 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:56.571724 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:56.572257 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:56.573149 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:56.573370 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:56.573450 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:56.573695 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:56.574089 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:56.574176 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:56.574269 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 03:29:56.574317 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":88,"threads_started":1}
I20260809 03:29:56.581732 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:56.577714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.9.46.253:41429)
0809 03:29:56.577828 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:56.577833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:56.578038 (+ 205us) server_negotiation.cc:407] Connection header received
0809 03:29:56.578225 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:56.578228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:56.578272 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:56.578372 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:56.578641 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:56.579331 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:56.580474 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:56.580838 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:56.580964 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:56.581201 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:56.581282 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:56.581362 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:56.581506 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 03:29:56.581569 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 03:29:56.582345 1707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:56.578394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55512 (local address 127.9.46.252:44357)
0809 03:29:56.578502 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:56.578506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:56.578518 (+ 12us) server_negotiation.cc:407] Connection header received
0809 03:29:56.578561 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:56.578566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:56.578627 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:56.578722 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:56.578960 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:56.579679 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:56.580786 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:56.581079 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:56.581193 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:56.581861 (+ 668us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:56.581942 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:56.582010 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:56.582171 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 03:29:56.582223 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 03:29:56.582847 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 03:29:56.578179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.9.46.254:37371)
0809 03:29:56.578444 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 03:29:56.578449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 03:29:56.578463 (+ 14us) server_negotiation.cc:407] Connection header received
0809 03:29:56.578547 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 03:29:56.578549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 03:29:56.578723 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 03:29:56.578838 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 03:29:56.579193 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:56.579917 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:56.580993 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 03:29:56.581297 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 03:29:56.581404 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 03:29:56.581913 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 03:29:56.582306 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 03:29:56.582582 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 03:29:56.582629 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 03:29:56.582701 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
I20260809 03:29:56.585830 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 1318
I20260809 03:29:56.596045 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 1395
I20260809 03:29:56.603430 9403 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOU4Lix/build/debug/bin/kudu with pid 1477
2026-08-09T03:29:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3181 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6363 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (292467 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST